search

pypest / pyemu

python modules for model-independent uncertainty analyses, data-worth analyses, and interfacing with PEST(++)
BSD 3-Clause "New" or "Revised" License
169 stars 94 forks source link

Parameter and Observation ordering #409

Closed Daniel-Trung-Nguyen closed 10 months ago

Daniel-Trung-Nguyen commented 1 year ago

Hi, I found the PstFrom organize parameters and observations in an arbitrary order making it harder to update them using pandas. Is there a way to align them with the order in the observation and parameter files so I can do column swapping easily? Many thanks, Daniel

image image

image

wkitlasten commented 1 year ago

I always keep a unique identifier for all table based pars and obs and use that as my pst_from index_col. Then I can easily split that off and join to other tables.

In your case, is pst _from assigning the name "ob10" to the 1987 sample?

Daniel-Trung-Nguyen commented 1 year ago

Yes, here is the longname version of that. You can see the first obs is at the 10th location and the time series data is jumping around. I am using pyemu shipped with the GMDSI_notebooks

image

wkitlasten commented 1 year ago

What does your pf.add_parameters() call look like?

jtwhite79 commented 1 year ago

Im sure there are places where we are using sets/dicts for lookup speed which is why the order is not aligned with the interface files.

briochh commented 10 months ago

Hi @Daniel-Trung-Nguyen, Suspect that you are passed this issue now. We are definitely making use for sets and dicts for lookup speed. This can disrupt ordering. This can be a bit of a gotcha but we generally align things (pars and obs) on the values of indicies. As a general rule, safer to slice and dice those dataframes using loc rather than iloc (and assuming some preservation of order). Will close this issue for now, reopen if still causing problems.