encountered an error with pestpp

[liufan612]

I made a test model to test whether the parameter estimation could succeed, but I encountered an error。The following error occured： （exception raised by run thread: thread processing instruction file raised an exception: InstructionFile error in file 'heads.csv.ins' : output file'heads.csv' not found exception raised by run thread: thread processing instruction file raised an exception: InstructionFile error in file 'heads.csv.ins' : output file'heads.csv' not found Error running model: Invalid argument Aborting model run） The file structure of the mode and code l I refer to pstfrom_mf6.ipynb，so I don't think it's a path problem. And I moved pstfrom_mf6.ipynb to run in the liunx environment, and I also got the same error, and I couldn't find the output file corresponding to the instruction file.

[mnfienen]

Hi @liufan612

This usually indicates that the forward model (e.g. MF6) failed to run. I recommend setting NOPTMAX=0 and running the model once in a copy of the full run directory to see if there is any screen output about the model failing

[liufan612]

Thanks, based on your suggestion, I've fixed the issue running in liunx due to forward_run.py not working properly. But in the window environment, my own test model, ran the model correctly, but still got the above error.The complete operation log is as follows： MODFLOW-2005
U.S. GEOLOGICAL SURVEY MODULAR FINITE-DIFFERENCE GROUND-WATER FLOW MODEL Version 1.12.00 2/3/2017

Using NAME file: test.nam Run start date and time (yyyy/mm/dd hh:mm:ss): 2023/05/10 11:47:58

Solving: Stress period: 1 Time step: 1 Ground-Water Flow Eqn. Run end date and time (yyyy/mm/dd hh:mm:ss): 2023/05/10 11:47:58 Elapsed run time: 0.042 Seconds

Normal termination of simulation 05/10/23 11:47:58 foward run command(s) finished, took 2.011 seconds 05/10/23 11:47:58 processing instruction files with 1 threads... exception raised by run thread: thread processing instruction file raised an exception: InstructionFile error in file 'test.hob.out.ins' : output file'test.csv' not found

exception raised by run thread: thread processing instruction file raised an exception: InstructionFile error in file 'test.hob.out.ins' : output file'test.csv' not found

Error running model: invalid argument: invalid argument Aborting model run

-->05/10/23 11:47:58 run complete, took: 2.017 seconds -->0 of 1 complete, 1 failed

[wkitlasten]

Hi, have you confirmed if the 'test.csv' is being created by your forward_run.py?

On Wed, May 10, 2023 at 3:58 PM liufan612 @.***> wrote:

Thanks, based on your suggestion, I've fixed the issue running in liunx due to forward_run.py not working properly. But in the window environment, my own test model, ran the model correctly, but still got the above error.The complete operation log is as follows： MODFLOW-2005 U.S. GEOLOGICAL SURVEY MODULAR FINITE-DIFFERENCE GROUND-WATER FLOW MODEL Version 1.12.00 2/3/2017

Using NAME file: test.nam Run start date and time (yyyy/mm/dd hh:mm:ss): 2023/05/10 11:47:58

Solving: Stress period: 1 Time step: 1 Ground-Water Flow Eqn. Run end date and time (yyyy/mm/dd hh:mm:ss): 2023/05/10 11:47:58 Elapsed run time: 0.042 Seconds

Normal termination of simulation 05/10/23 11:47:58 foward run command(s) finished, took 2.011 seconds 05/10/23 11:47:58 processing instruction files with 1 threads... exception raised by run thread: thread processing instruction file raised an exception: InstructionFile error in file 'test.hob.out.ins' : output file'test.csv' not found

exception raised by run thread: thread processing instruction file raised an exception: InstructionFile error in file 'test.hob.out.ins' : output file'test.csv' not found

Error running model: invalid argument: invalid argument Aborting model run

-->05/10/23 11:47:58 run complete, took: 2.017 seconds -->0 of 1 complete, 1 failed

— Reply to this email directly, view it on GitHub https://github.com/pypest/pyemu/issues/427#issuecomment-1541319177, or unsubscribe https://github.com/notifications/unsubscribe-auth/ADSJXRGNBIEVDK3NMZZAR7DXFMHAFANCNFSM6AAAAAAX3GUDCM . You are receiving this because you are subscribed to this thread.Message ID: @.***>

-- "Perfect spheres are pointless."

[liufan612]

Sorry, I don't understand. According to forward run.py, which runs the model successfully, I don't see any attempt to recreate the output file corresponding to the instruction file. This is forward run.py of the successful run example forward run.py, where is the output file created? forward run.py: import os import multiprocessing as mp import numpy as np import pandas as pd import pyemu import flopy def main(): try: os.remove(r'sfr.csv') except Exception as e: print(r'error removing tmp file:sfr.csv') try: os.remove(r'heads.csv') except Exception as e: print(r'error removing tmp file:heads.csv') pyemu.helpers.apply_list_and_array_pars(arr_par_file='mult2model_info.csv',chunk_len=50) pyemu.os_utils.run(r'mf6') if name == 'main': mp.freeze_support() main()

[wkitlasten]

What does your pf.add_observations() call look like?

Your error:

'test.hob.out.ins' : output file'test.csv' not found

Implies pest is looking for a file called 'test.csv' that has simulation results, and that 'test.hob.out.ins' contains the "instructions" to read that file. But 'test.csv' isn't being created. Are you creating a 'test.hob.out' file by chance? Does the 'test.hob.out.ins' file contain the appropriate instructions to read the 'test.hob.out' file?

On Wed, May 10, 2023 at 8:31 PM liufan612 @.***> wrote:

Sorry, I don't understand. According to forward run.py, which runs the model successfully, I don't see any attempt to recreate the output file corresponding to the instruction file. This is forward run.py of the successful run example forward run.py, where is the output file created? forward run.py: import os import multiprocessing as mp import numpy as np import pandas as pd import pyemu import flopy def main(): try: os.remove(r'sfr.csv') except Exception as e: print(r'error removing tmp file:sfr.csv') try: os.remove(r'heads.csv') except Exception as e: print(r'error removing tmp file:heads.csv')

pyemu.helpers.apply_list_and_array_pars(arr_par_file='mult2model_info.csv',chunk_len=50) pyemu.os_utils.run(r'mf6') if name == 'main': mp.freeze_support() main()

— Reply to this email directly, view it on GitHub https://github.com/pypest/pyemu/issues/427#issuecomment-1541574557, or unsubscribe https://github.com/notifications/unsubscribe-auth/ADSJXRHZNUXOEDR2KSWKNV3XFNG4ZANCNFSM6AAAAAAX3GUDCM . You are receiving this because you commented.Message ID: @.***>

-- "Perfect spheres are pointless."

[liufan612]

I made some changes as suggested, but it still didn't work. Comparing the sample files in the tutorial, I found one with the obs suffix. Is this file required? Is it generated by FLOPY or pyemu? I used GMS to perform parameter estimation, and found that this obs format file would also be generated, so I think the problem may lie here.

[mnfienen]

@liufan612 - the OBS file is made directly by MODFLOW 6. GMS may be set up to use a different method to extract modelled values from the results. so likely you will need to associate your ins file with test.hob.out rather than test.csv. If you upload your notebook, it will be easier to find where things maybe got off track....

[briochh]

@liufan612, did you get to the bottom of this issue? Looks like the output file that PEST was expecting to find wasn't being outputted as part of the forward model run.

Closing this issue, but reopen if still a problem