search

pypest / pyemu

python modules for model-independent uncertainty analyses, data-worth analyses, and interfacing with PEST(++)
BSD 3-Clause "New" or "Revised" License
176 stars 96 forks source link

Failing to process my list files? #534

Open wkitlasten opened 2 months ago

wkitlasten commented 2 months ago

I'm failing on line 2339 in helpers with a previously working setup.

mlts[mlt.index_cols] = mlts[mlt.index_cols].apply(
                _try_pdcol_numeric, intadj=add1, downcast='integer')

KeyError: "None of [Index(['ifno'], dtype='object')] are in the [columns]"

mlts is:

Unnamed: 0 sidx idx_strs pargpvalue parval1_value value 0 0 (1109,) ifno:1109 sfr.inflow 1.0 1.0 1 1 (1134,) ifno:1134 sfr.inflow 1.0 1.0 2 2 (1160,) ifno:1160 sfr.inflow 1.0 1.0 3 3 (1168,) ifno:1168 sfr.inflow 1.0 1.0 4 4 (1177,) ifno:1177 sfr.inflow 1.0 1.0 5 5 (1193,) ifno:1193 sfr.inflow 1.0 1.0 6 6 (1286,) ifno:1286 sfr.inflow 1.0 1.0 7 7 (1297,) ifno:1297 sfr.inflow 1.0 1.0 8 8 (1324,) ifno:1324 sfr.inflow 1.0 1.0 9 9 (1337,) ifno:1337 sfr.inflow 1.0 1.0 10 10 (1364,) ifno:1364 sfr.inflow 1.0 1.0 11 11 (1365,) ifno:1365 sfr.inflow 1.0 1.0

and mlt.index_cols = ['ifno']

So it appears to be looking for ifno in the mlts dataframe, but it should be looking for ifno in the org file (ifno is the correct column name in the org file but not the mlts df).

Is there a change that requires me to rebuild everything? Is this section of code working for everyone else (tests), or might there a mistake here? Any idea what I might have messed up somewhere along the way? (I don't think I touched this part of the setup, but I never know!)

jtwhite79 commented 2 months ago

Have you updated pyemu or pandas or numpy recently? There have been some changes...

briochh commented 2 months ago

@wkitlasten sounds like you are using a update pyemu to apply parameters in an existing pest interface built with an older version? This is definitely a space where things can go wrong. It is a blind spot in testing too as we can only efficiently test setup and apply with the same consistent version. And as @jtwhite79 says there have been changes in that area recently with numpy 2.0. If you don't want to rebuild your interface the option might be to drop pyemu back a few versions.

briochh commented 2 months ago

Would be keen to check that file though, if you wanna send it through with the add par(s) calls you are using.

wkitlasten commented 2 months ago

Thanks J. I had just updated pyemu, but not pandas (scarred) or numpy (scared). I reverted back to an older pyemu to keep moving. @briochh I’ll send the file(s) today.

gotta stay focused!! image