transport_data.util.sdmx.CSVAdapter¶

class transport_data.util.sdmx.CSVAdapter(path: pathlib.Path, structure: str | None = None, structure_id: str | None = None, action: str | None = None)[source]¶

Bases: RawIOBase

Adapt CSV content from path into SDMX-CSV.

The SDMX-CSV format has precise requirements for identifier and other columns. In particular, the “STRUCTURE”, “STRUCTURE_ID”, and “ACTION” columns are mandatory.

STRUCTURE,STRUCTURE_ID,ACTION,DIM_1,DIM_2,DIM_3,OBS_VALUE
dataflow,ESTAT:NA_MAIN(1.6.0),I,A,B,2014-01,12.4

This class produces standard SDMX-CSV by adapting a simplified format such as:

DIM_1,DIM_2,DIM_3,OBS_VALUE
A,B,2014-01,12.4

In particular:

The header line of path is prefixed with “STRUCTURE”, “STRUCTURE_ID”, and/or “ACTION” if the respective parameters are given.
Every record (line) of path is prefixed with the values of the respective parameters.

So, for instance, to transform the above example into the first example, use a CSVAdapter with:

a = CSVAdapter(
    path,
    structure="dataflow",
    structure_id="ESTAT:NA_MAIN(1.6.0)",
    action="I",
)
data = a.read()

Parameters:

structure – Value for the “STRUCTURE” SDMX-CSV field, to be inserted into every record.
structure_id – Value for the “STRUCTURE_ID” field, to be inserted into every record.
action – Value for the “ACTION” field, to be inserted into every record.

__init__(path: pathlib.Path, structure: str | None = None, structure_id: str | None = None, action: str | None = None) → None¶

Methods

`__init__`(path[, structure, structure_id, action])
`close`()	Flush and close the IO object.
`fileno`()	Return underlying file descriptor if one exists.
`flush`()	Flush write buffers, if applicable.
`isatty`()	Return whether this is an 'interactive' stream.
`read`([size])
`readable`()	Return whether object was opened for reading.
`readall`([size])	Read and adapt CSV to SDMX-CSV.
`readinto`
`readline`([size])	Read and return a line from the stream.
`readlines`([hint])	Return a list of lines from the stream.
`seek`(offset[, whence])	Change the stream position to the given byte offset.
`seekable`()	Return whether object supports random access.
`tell`()	Return current stream position.
`truncate`([size])	Truncate file to size bytes.
`writable`()	Return whether object was opened for writing.
`write`
`writelines`(lines, /)	Write a list of lines to stream.

Attributes

closed

close()¶

Flush and close the IO object.

This method has no effect if the file is already closed.

fileno()¶

Return underlying file descriptor if one exists.

Raise OSError if the IO object does not use a file descriptor.

flush()¶

Flush write buffers, if applicable.

This is not implemented for read-only and non-blocking streams.

isatty()¶

Return whether this is an ‘interactive’ stream.

Return False if it can’t be determined.

readable()¶

Return whether object was opened for reading.

If False, read() will raise OSError.

readall(size: int = -1, /) → bytes¶: Read and adapt CSV to SDMX-CSV.

Todo

This currently reads and adapts the entire file at once; this may not be performant for very large files. Improve to handle lines individually or in batches.

readline(size=-1, /)¶

Read and return a line from the stream.

If size is specified, at most size bytes will be read.

The line terminator is always b’n’ for binary files; for text files, the newlines argument to open can be used to select the line terminator(s) recognized.

readlines(hint=-1, /)¶

Return a list of lines from the stream.

hint can be specified to control the number of lines read: no more lines will be read if the total size (in bytes/characters) of all lines so far exceeds hint.

seek(offset, whence=0, /)¶

Change the stream position to the given byte offset.

offset
The stream position, relative to ‘whence’.

whence
The relative position to seek from.

The offset is interpreted relative to the position indicated by whence. Values for whence are:

os.SEEK_SET or 0 – start of stream (the default); offset should be zero or positive
os.SEEK_CUR or 1 – current stream position; offset may be negative
os.SEEK_END or 2 – end of stream; offset is usually negative

Return the new absolute position.

seekable()¶

Return whether object supports random access.

If False, seek(), tell() and truncate() will raise OSError. This method may need to do a test seek().

tell()¶: Return current stream position.

truncate(size=None, /)¶

Truncate file to size bytes.

File pointer is left unchanged. Size defaults to the current IO position as reported by tell(). Return the new size.

writable()¶

Return whether object was opened for writing.

If False, write() will raise OSError.

writelines(lines, /)¶

Write a list of lines to stream.

Line separators are not added, so it is usual for each of the lines provided to have a line separator at the end.