The archiver is currently fairly confusing, which makes it difficult to add new datasets. Refactoring this workflow so there is only one place where everything related to a single dataset is defined could make this process easier. In the refactoring process, we can also simplify some of the general logic flow and improve logging functionality to make debugging the archiver much easier.
zschira
commented
1 year ago
The archiver is currently fairly confusing, which makes it difficult to add new datasets. Refactoring this workflow so there is only one place where everything related to a single dataset is defined could make this process easier. In the refactoring process, we can also simplify some of the general logic flow and improve logging functionality to make debugging the archiver much easier.