christopher-mangion
commented
4 months ago - Removed SystemManager in favour of an individual manager for each of downloading, processing, and conversion
- Renamed old functional scripts into newer names (download, process, convert)
- Config files updated to be split into downloading, processing, and conversion stages
- Added subsections option to config
- sources.py now contains both SourceManager and Location (renamed from SourceLocation, removed location.py)
- ArgParser now grabs prepare, quiet, and subsection variables
- ArgParser no longer has file select (unnecessary)
- Changed default database to be URL database
- Changed Crawler to now ignore saved progress when overwriting for crawl database
- Changed multiple nucleotide dirs into one dir with subsections
- Combined wilderlab into one source with subsections
- Updated all config files to fit new format
- Updated README