CADWRDeltaModeling / dms_datastore

Data download and management tools for continuous data for Pandas. See documentation https://cadwrdeltamodeling.github.io/dms_datastore/
https://cadwrdeltamodeling.github.io/dms_datastore/
MIT License
1 stars 0 forks source link

assemble_dataframe #44

Open water-e opened 5 months ago

water-e commented 5 months ago

I believe this has been done many times in different contexts. One example is hotstart_nudging_data.py in BayDeltaSCHISM. I think @sophunger had another use case. @dwr-psandhu I think you did this with dask, although you haven't been pushing dash as hard lately.

The idea is to be able to programmatically assemble a dataframe from a start, end, variable and list of of different stations. It is a bit easier if we assume they are all the same variable, but if we can put up with a multiindex for the columns we could do station and variable. Sublocations would use the @ sign because I think multiindexing those would be tough. Or they could be yet another level.

Fine with me if this is limited to screened data with usage as simple as read_flagged. I know dask will never work without uniform formatting.