Offload directory scanning

WIPACrepo / file_catalog

Store file metadata information in a file catalog

MIT License

1 stars 4 forks source link

Closed ric-evans closed 4 years ago

ric-evans commented 4 years ago

moved the /data/exp filepath collection process from indexer_make_dag.py to allpaths.py and all_paths_make_condor.py
now in order to index all of /data/exp:
1. run all_paths_make_condor.py
2. run indexer_make_dag.py with flag pointing to directory produced from allpaths.py

dsschult commented 4 years ago

I'm ok with "squash and merge" now.