Refactoring background ingestor.

SciCatProject / scicat-filewriter-ingest

Python client that connects to a kafka queue and creates new datasets when receiving notification that a file has been written

0 stars 0 forks source link

Closed YooSunYoung closed 1 week ago

YooSunYoung commented 2 weeks ago

Renamed the background ingestor script so that it can be used as a module as well.
Extracted the system helper function (exiting at exceptions) to a separate helper module
- The scripts can be more concentrating on the logic.
- Avoid duplicating code
Add how to install/run in the readme both, online/background ingestor.

Next step will be adding functionalities into the background ingestor and add tests.