search

SoilPulse / MetadataGenerator

The metadata generator guides researchers to the creation of reusable data sets in a webapp.
GNU General Public License v3.0
0 stars 0 forks source link

handling of downloaded files #19

Closed jonaslenz closed 3 months ago

jonaslenz commented 6 months ago

Hi Honza, I see a possible issue with the file handling.

Right now the files loaded (and extractet) by the ressource manager are stored in the folder "/downloaded_files/". When adding another ressource the files are put in the same folder and might overwrite a file of another dataset, when having the same filename?

The mapping of the files within the ressource manager nevertheless seems to work fine (>>RM.showContainerTree() is not affected by foreign files).

I suggest to create a separate folder for each data ressource (i.e. DOI) to avoid overwritting. And would you mind to put the data in the "/catalog/", instead of "/downloaded_files/", or add "/downloaded_files/"to gitignore.

jandevaty commented 6 months ago

yes, I have considered that - as it is right now one ResourceManagers files should be all disscarded after the session not to infringe anyones rights, and only after literal consent we will move the files to permanent storage. The ResourceManager can be established in the central database as soon as it will be created at the start of the session (or loaded if editing existing), so it will receive an unique ID that will be used all through - including naming a root dictionary for this RM instance.

jonaslenz commented 6 months ago

yes, I have considered that - as it is right now one ResourceManagers files should be all disscarded after the session not to infringe anyones rights, and only after literal consent we will move the files to permanent storage.

We might depent this behaviour on the licence assigned to the dataset.

The ResourceManager can be established in the central database as soon as it will be created at the start of the session (or loaded if editing existing), so it will receive an unique ID that will be used all through - including naming a root dictionary for this RM instance.

Sounds great!

jandevaty commented 3 months ago

Solved as suggested - each Project (former ResourceManager) gets unique ID assigned and gets own directory (with the ID as name) for files.