SwissOpenEM / ScopeMArchiver

0 stars 0 forks source link

[Epic] Ingestion Service #32

Open consolethinks opened 3 months ago

consolethinks commented 3 months ago

An ingestion service mode that would be used by universities, institutions to transfer their datasets to a central instance of SciCat with a corresponding central archival system. This would be achieved through a service that would be accessing datasets from an on-site location, it would provide a way to register them in SciCat by filling out metadata (both by manual input and automatic retrieval), then transfer them to the cache server of the archival system. After transfer, the datasets would be marked as "archivable".

These jobs should be executed one by one in order to maximize throughput, and should follow a FIFO order, maybe with a few priority levels.

Features: