This PR adds a base class ascat.file_handling.Filenames which handles operations on filenames that are passed to it. It assumes child classes implement the needed methods out of: _read (for reading a single file), _write (for writing data to a single file), and _merge (for merging a list of data objects into a single one). It then handles the reading, writing, and merging of >=1 files using the subclass methods.
The PR also converts all the level 1 and 2 readers to Filenames subclasses along these lines. In fact, they are subclasses of a more general ascat.read_native.AscatFile subclass, which overrides the Filenames read method with some extra logic for spatio-temporal subsetting.
Some things I'm not sure on yet -
Should subsetting be done on each file individually before merging or on the entire merged dataset after its read? I feel that the former makes more sense in terms of memory overhead but perhaps incurs unreasonable computational cost.
EDIT: Decided to do after read for now. May come back to this later.
I had to add a read_generic init argument to the AscatFile class, which determines the default value for generic in the read method of an AscatFile object if read is not passed a value for generic when called. This is to reproduce the behavior of the old AscatL1bFile and AscatL2File classes, which instantiated reader classes and always used their read methods with generic=True by default, whereas the reader classes themselves had a default of generic=False. Perhaps there is a better solution though?
EDIT: Solved this by creating an, e.g., AscatL1bEpsFileGeneric class for every product, which simply redefines the _read method with generic=True as a default argument. These are the classes that AscatL1bFile and AscatL2File use to instantiate readers.
This PR adds a base class
ascat.file_handling.Filenames
which handles operations on filenames that are passed to it. It assumes child classes implement the needed methods out of:_read
(for reading a single file),_write
(for writing data to a single file), and_merge
(for merging a list of data objects into a single one). It then handles the reading, writing, and merging of >=1 files using the subclass methods.The PR also converts all the level 1 and 2 readers to Filenames subclasses along these lines. In fact, they are subclasses of a more general
ascat.read_native.AscatFile
subclass, which overrides theFilenames
read method with some extra logic for spatio-temporal subsetting.Some things I'm not sure on yet -
Should subsetting be done on each file individually before merging or on the entire merged dataset after its read? I feel that the former makes more sense in terms of memory overhead but perhaps incurs unreasonable computational cost.
I had to add a
read_generic
init argument to theAscatFile
class, which determines the default value forgeneric
in theread
method of anAscatFile
object ifread
is not passed a value forgeneric
when called. This is to reproduce the behavior of the oldAscatL1bFile
andAscatL2File
classes, which instantiated reader classes and always used theirread
methods withgeneric=True
by default, whereas the reader classes themselves had a default ofgeneric=False
. Perhaps there is a better solution though?AscatL1bEpsFileGeneric
class for every product, which simply redefines the_read
method withgeneric=True
as a default argument. These are the classes thatAscatL1bFile
andAscatL2File
use to instantiate readers.