Open awst-baum opened 5 years ago
maybe add an option to allow user to select where to store/load the metadata from?
Generally I think the metadata handling could be improved, e.g raising a meaningful error when the data has been moved and the stored paths are not valid anymore.
I would just start with relative paths. That should be pretty straightforward to implement.
Custom metadata path might be a little bit trickier.
The ISMN reader seems to create a cache of the ISMN data in a folder in a subfolder called
python_metadata
in a file calledmetadata.npy
.metadata.npy
contains absolute pathnames of the stm files, e.g./path/to/mydata/ISMN/ISMN_V20180830_GLOBAL/AMMA-CATCH/Banizoumbou/AMMA-CATCH_AMMA-CATCH_Banizoumbou_sm_0.050000_0.050000_CS616-1_19780101_20180830.stm
.In our setup, we use the same NFS mount containing ISMN data in several systems, and not all systems use the same mountpoint (e.g. one might have the data at
/path/to/mydata
and the other have it at/some/completely/different/path/to/my/data
). Also, we use linux softlinks, so we might also access the data as/softlink/to/data/...
.Once the metadata folder is created by one of the systems, the other systems may produce errors if they don't use the same path to access the data because they can't find the stm files at the paths cached in the
metadata.npy
file.Would it be possible to not use absolute file paths or make it possible in some other way to use the same ISMN folder from several systems?