Ultimately we want to change db_load.py so that we aren't redundantly creating a bunch of YAMLs that need to be parsed and can streamline arguments to pass in for a later crawl script
Shell script, pass directory, in shell script find metadata.yaml based on parent directory, pass that onto qcdb & pass directory for qcdb to search through
params.yaml - change from data directory to just tools
db_load.py - change to run tool parsers based on params.yaml, run on data directory based on passed in argument, run on metadata directory based on passed in argument
aguang
commented
2 years ago
Ultimately we want to change
db_load.pyso that we aren't redundantly creating a bunch of YAMLs that need to be parsed and can streamline arguments to pass in for a later crawl scriptShell script, pass directory, in shell script find metadata.yaml based on parent directory, pass that onto qcdb & pass directory for qcdb to search through
params.yaml - change from data directory to just tools
db_load.py - change to run tool parsers based on params.yaml, run on data directory based on passed in argument, run on metadata directory based on passed in argument