search

OGGM / oggm

Open Global Glacier Model
http://oggm.org
BSD 3-Clause "New" or "Revised" License
217 stars 104 forks source link

Lack ‘inversion_input.pkl’ files #1721

Closed y624745579 closed 3 weeks ago

y624745579 commented 1 month ago

Helllo, when I tried the thickness inversion tutorial [https://oggm.org/tutorials/stable/notebooks/beginner/inversion.htm] and I changed "rgi_region = '11' " to rgi_region" to 15，but I found that 'inversion_input.pkl, inversion_output.pkl, inversion_flowlines.pkl' were missing in many glacier directories compared with orther directories. I checked the pre-downloaded files on the server and found that these files were indeed missing . How should I deal with this error? Thank you! The output is : 2024-08-08 16:30:57: oggm.cfg: Reading default parameters from the OGGM params.cfg configuration file. 2024-08-08 16:30:57: oggm.cfg: Multiprocessing switched OFF according to the parameter file. 2024-08-08 16:30:57: oggm.cfg: Multiprocessing: using all available processors (N=16) 2024-08-08 16:30:58: oggm.cfg: Multiprocessing switched ON after user settings. 2024-08-08 16:30:58: oggm.cfg: PARAMS['continue_on_error'] changed from False to True. 2024-08-08 16:30:59: oggm.workflow: init_glacier_directories from prepro level 3 on 4238 glaciers. 2024-08-08 16:30:59: oggm.workflow: Execute entity tasks [gdir_from_prepro] on 4238 glaciers [0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1, 1.1, 1.2, 1.3, 1.5, 1.7, 2, 2.5, 3, 4, 5, 6, 7, 8, 9, 10] 2024-08-08 16:31:40: oggm.core.inversion: FileNotFoundError occurred during task mass_conservation_inversion on RGI60-15.03706: [Errno 2] No such file or directory: '/home/OGGM/Inversion_thk/per_glacier/RGI60-15/RGI60-15.03/RGI60-15.03706/inversion_input.pkl' 2024-08-08 16:31:41: oggm.core.inversion: FileNotFoundError occurred during task mass_conservation_inversion on RGI60-15.10450: [Errno 2] No such file or directory: '/home/OGGM/Inversion_thk/per_glacier/RGI60-15/RGI60-15.10/RGI60-15.10450/inversion_input.pkl' 2024-08-08 16:33:46: oggm.core.inversion: FileNotFoundError occurred during task mass_conservation_inversion on RGI60-15.03706: [Errno 2] No such file or directory: '/home/OGGM/Inversion_thk/per_glacier/RGI60-15/RGI60-15.03/RGI60-15.03706/inversion_input.pkl' 2024-08-08 16:33:48: oggm.core.inversion: FileNotFoundError occurred during task mass_conservation_inversion on RGI60-15.10450: [Errno 2] No such file or directory: '/home/OGGM/Inversion_thk/per_glacier/RGI60-15/RGI60-15.10/RGI60-15.10450/inversion_input.pkl'

anoukvlug commented 1 month ago

@y624745579, as I don't know the details of the issue you're encountering, it is hard to give advise (e.g. how many glacier are missing of the 4238 you're trying to run? Which pre-processed directories are you using?). The following info in the OGGM documentation on the topic of errors might be useful to you. More generally I would advice you to read the OGGM error guide blog post. Questions about OGGM usage can best be asked in the OGGM slack support channel.

y624745579 commented 1 month ago

@y624745579, as I don't know the details of the issue you're encountering, it is hard to give advise (e.g. how many glacier are missing of the 4238 you're trying to run? Which pre-processed directories are you using?). The following info in the OGGM documentation on the topic of errors might be useful to you. More generally I would advice you to read the OGGM error guide blog post. Questions about OGGM usage can best be asked in the OGGM slack support channel.

@anoukvlug, thank you for your patient reply! My question is very simple, that is, I found that some of the pre-processed data download from the OGGM shop lack some inversion files such as inversion_input.pkl, etc., which are necessary files for ice thickness inversion. For example, my research area happens to be RGI60-15.03706, which lack inversion files, can I generate the corresponding inversion files myself ? I didn't find these in the OGGM tutorial.

The source code is: from oggm import cfg, utils, workflow, tasks, graphics, DEFAULT_BASE_URL cfg.initialize(logging_level='WARNING') cfg.PATHS['working_dir'] = utils.gettempdir(dirname='/home/link/OGGM/test_inversion', reset=True) rgi_ids = ['RGI60-15.03706']

base_url = base_url = ('https://cluster.klima.uni-bremen.de/~oggm/gdirs/oggm_v1.6/' 'L3-L5_files/2023.3/centerlines/W5E5/') glen_a = 2.4e-24 fs = 5.7e-20 gdirs = workflow.init_glacier_directories(rgi_ids, from_prepro_level=3, prepro_base_url=base_url) workflow.execute_entity_task(tasks.mass_conservation_inversion, gdirs, glen_a=glen_a, fs=fs) , and the output is: 2024-08-08 20:33:07: oggm.cfg: Reading default parameters from the OGGM params.cfg configuration file. 2024-08-08 20:33:07: oggm.cfg: Multiprocessing switched OFF according to the parameter file. 2024-08-08 20:33:07: oggm.cfg: Multiprocessing: using all available processors (N=16) 2024-08-08 20:33:11: oggm.workflow: init_glacier_directories from prepro level 3 on 1 glaciers. 2024-08-08 20:33:11: oggm.workflow: Execute entity tasks [gdir_from_prepro] on 1 glaciers 2024-08-08 20:33:11: oggm.workflow: Execute entity tasks [mass_conservation_inversion] on 1 glaciers 2024-08-08 20:33:11: oggm.core.inversion: FileNotFoundError occurred during task mass_conservation_inversion on RGI60-15.03706: [Errno 2] No such file or directory: '/home/link/OGGM/test_inversion/per_glacier/RGI60-15/RGI60-15.03/RGI60-15.03706/inversion_input.pkl' Traceback (most recent call last): File "/home/link/OGGM/prepare_input.py", line 13, in workflow.execute_entity_task(tasks.mass_conservation_inversion, gdirs, File "/home/link/app/anaconda3/envs/oggm_env/lib/python3.12/site-packages/oggm/workflow.py", line 191, in execute_entity_task out = [pc(gdir) for gdir in gdirs] ^^^^^^^^ File "/home/link/app/anaconda3/envs/oggm_env/lib/python3.12/site-packages/oggm/workflow.py", line 108, in call res = self._call_internal(func, arg, kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/link/app/anaconda3/envs/oggm_env/lib/python3.12/site-packages/oggm/workflow.py", line 102, in _call_internal return call_func(gdir, kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/link/app/anaconda3/envs/oggm_env/lib/python3.12/site-packages/oggm/utils/_workflow.py", line 492, in _entity_task out = task_func(gdir, kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/link/app/anaconda3/envs/oggm_env/lib/python3.12/site-packages/oggm/core/inversion.py", line 410, in mass_conservation_inversion cls = gdir.read_pickle('inversion_input') ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/link/app/anaconda3/envs/oggm_env/lib/python3.12/site-packages/oggm/utils/_workflow.py", line 3151, in read_pickle with _open(fp, 'rb') as f: ^^^^^^^^^^^^^^^ File "/home/link/app/anaconda3/envs/oggm_env/lib/python3.12/gzip.py", line 61, in open binary_file = GzipFile(filename, gz_mode, compresslevel) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/link/app/anaconda3/envs/oggm_env/lib/python3.12/gzip.py", line 192, in init fileobj = self.myfileobj = builtins.open(filename, mode or 'rb') ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ FileNotFoundError: [Errno 2] No such file or directory: '/home/link/OGGM/test_inversion/per_glacier/RGI60-15/RGI60-15.03/RGI60-15.03706/inversion_input.pkl'

anoukvlug commented 1 month ago

@y624745579, please read the following Some glaciers exit with errors. What should I do? in the FAQ and Troubleshooting of the OGGM documentation. It really depends on what the reason is that the files are missing, whether it will be straightforward to generate these files yourself and if it would be worth the effort to solve.

Depending on the error that caused the files to be missing, it might be hard to solve it. The log.txt file of 'RGI60-15.03706' shows that an error already occurred early on in the workflow, a couple of steps before the inversion, so that is where you would have to start. Anyway as this issue is clearly a model usage question (and not related to e.g. model development), please ask a follow-up question you might have on the OGGM slack instead.