search

CADWRDeltaModeling / dms_datastore

Data download and management tools for continuous data for Pandas. See documentation https://cadwrdeltamodeling.github.io/dms_datastore/
https://cadwrdeltamodeling.github.io/dms_datastore/
MIT License
1 stars 0 forks source link

process_* are related to Bay Delta Schism and should be moved there #9

Open dwr-psandhu opened 10 months ago

dwr-psandhu commented 10 months ago

Description

There are process_*.py that transform the repo data into schism related files. These should be moved from here to BayDeltaSCHISM repo

https://github.com/CADWRDeltaModeling/BayDeltaSCHISM

water-e commented 5 months ago

@dwr-psandhu It would be worth drilling into this. I agree the *.th format and particular combinations for SCHISM are not of general interest. Yet the scripts contain things that might be general. Things like filled time series or even more complex combinations as in process_yolo.py or process_exports.py? Where will we put processed data (or derived)?

Should the functionality be in dms_datastore or a new dms_dataprep? I think we will need to assume some useful data production efforts are federated and done outside then added to the repository on a dropbox-like basis and I'm working sloly on that.

However would make sense to force/centralize the issue a bit for the really important stuff like the boundaries? I don't mind that these products be stashed away in univariate csv format. Then later on the process_*.py scripts in BayDeltaSCHISM would do a much smaller workload -- just pick up the already-done work and put it in the th format.