(Meta)data storage

transport_data.store provides for storing and handling two kinds of data files:

Local data

Data that exist only on an individual user’s system. These may include:

• Cached files such as those downloaded from particular network locations.

• Temporary or intermediate data files produced or used by other code in transport_data or downstream packages.

These data are handled by an instance of the LocalStore class.

Maintained data

These are data that may be distributed and referenced by other users and SDMX data or structures. They are handled by an instance of the Registry class.

transport_data.STORE is an instance of the UnionStore class, which dispatches operations to LocalStore or Registry according to the maintainer of objects. All three classes are subclassed of BaseStore and share a common interface.

Formats and layout

• Files are stored in the (configurable) directory indicated by Config.data_path. This defaults to the User data directory. For instance, on a Unix/Linux system, the data may be in $HOME/.local/share/transport-data/.

  • LocalStore files are in a …/local/ subdirectory.

  • Registry files are in a …/registry/ subdirectory.

• All files are stored as SDMX-ML (XML).

  Every SDMX MaintainableArtefact has a uniform resource name (URN) that resembles urn:sdmx:org.sdmx.infomodel.codelist.Codelist=TEST:COLOUR(1.2.3). In this example:

  • “TEST” is the id of a Agency stored as the maintainer attribute.

  • “COLOUR” is the ID of the object itself.

  • “(1.2.3)” is the version attribute of a VersionableArtefact.

• Directory and file names include maintainer ID, object class name, and object ID. For instance, the file TEST/Codelist_TEST_COLOUR.xml contains the Codelist with ID “COLOUR” maintained by the agency with ID “TEST”.

• Each file contains only objects of the class, ID, and maintainer corresponding to its path.

• Each file may contain multiple versions of objects. For instance, the file TEST/Codelist_TEST_COLOUR.xml may contain objects with the URNs and versions:

  • urn:sdmx:org.sdmx.infomodel.codelist.Codelist=TEST:COLOUR(1.0.0) → version 1.0.0;

  • urn:sdmx:org.sdmx.infomodel.codelist.Codelist=TEST:COLOUR(1.2.3) → version 1.2.3;

  • urn:sdmx:org.sdmx.infomodel.codelist.Codelist=TEST:COLOUR(1.2.4) → version 1.2.4;

  • and so on.

Code reference

Local data storage.

Todo

Add the following to base capabilities of UnionStore:

- :py:`write(..., annotate=True)`

Classes

UnionStore(config)

A store that maps between a LocalStore and Registry.