src/global_settings.py
This contains some of the global variables that may be used throughout the project
Makes the various path variables (e.g. path to the data directory) easier to access and readable
src/data/make_dataset.py
Now has simple validation of the preexisting scraped data
src/data/load_reports.py, src/data/mongodb.py
I simplified the model object so that new 'models' can be created easily. All you need for a new model is to define the FIELDS and COLLECTION_NAME and then in data_from_file() define how you will be extracting data from the file source.
Some python path setup troubleshooting in the readme
src/global_settings.pyThis contains some of the global variables that may be used throughout the project Makes the various path variables (e.g. path to the data directory) easier to access and readablesrc/data/make_dataset.pyNow has simple validation of the preexisting scraped datasrc/data/load_reports.py,src/data/mongodb.pyI simplified the model object so that new 'models' can be created easily. All you need for a new model is to define theFIELDSandCOLLECTION_NAMEand then indata_from_file()define how you will be extracting data from the file source.Some python path setup troubleshooting in the readme