search

E3SM-Project / e3sm_to_cmip

Tools to CMORize E3SM output
https://e3sm-to-cmip.readthedocs.io/en/latest/
MIT License
7 stars 7 forks source link

handler sftlf fails when processing long time series #208

Closed chengzhuzhang closed 1 year ago

chengzhuzhang commented 1 year ago

I ran into error for sftlf when processing 165 years of simulation data. Error message as follows:

2023-09-07 17:32:22,384_384:INFO:__init__:--------------------------------------
2023-09-07 17:32:22,384_384:INFO:__init__:| E3SM to CMIP Configuration
2023-09-07 17:32:22,384_384:INFO:__init__:--------------------------------------
2023-09-07 17:32:22,384_384:INFO:__init__:    * var_list='['sftlf']'
2023-09-07 17:32:22,384_384:INFO:__init__:    * input_path='/lcrc/group/e3sm/ac.zhang40/v3alpha02_cmorize_1850-2014/rgr/fixed_vars'
2023-09-07 17:32:22,384_384:INFO:__init__:    * output_path='/lcrc/group/e3sm/ac.zhang40/v3alpha02_cmorize_1850-2014'
2023-09-07 17:32:22,384_384:INFO:__init__:    * precheck_path='None'
2023-09-07 17:32:22,384_384:INFO:__init__:    * freq='mon'
2023-09-07 17:32:22,384_384:INFO:__init__:    * realm='fx'
2023-09-07 17:32:22,384_384:INFO:__init__:    * Writing log output file to: logs/20230907_173222_379856
2023-09-07 17:32:36,324_324:INFO:_get_handlers:--------------------------------------
2023-09-07 17:32:36,324_324:INFO:_get_handlers:| Derived CMIP6 Variable Handlers
2023-09-07 17:32:36,324_324:INFO:_get_handlers:--------------------------------------
2023-09-07 17:32:36,324_324:INFO:_get_handlers:    * 'sftlf' -> ['LANDFRAC']
2023-09-07 17:32:36,354_354:INFO:run:--------------------------------------
2023-09-07 17:32:36,355_355:INFO:run:| Running E3SM to CMIP in Parallel
2023-09-07 17:32:36,355_355:INFO:run:--------------------------------------
  0%|                                                                                                                | 0/1 [00:00<?, ?it/s]2023-09-07 17:32:36,425_425:INFO:handle_variables:sftlf: Starting
2023-09-07 17:32:36,618_618:INFO:handle_variables:sftlf: CMOR setup complete
2023-09-07 17:32:36,619_619:INFO:handle_variables:sftlf: loading LANDFRAC
2023-09-07 17:32:36,624_624:INFO:handle_variables:sftlf: loading axes
2023-09-07 17:32:36,625_625:INFO:handle_variables:sftlf: time 0.0 - 31.0
2023-09-07 17:32:36,725_725:INFO:handle_variables:sftlf: loading LANDFRAC
2023-09-07 17:32:36,729_729:INFO:handle_variables:sftlf: loading axes
2023-09-07 17:32:36,730_730:INFO:handle_variables:sftlf: time 31.0 - 59.0
2023-09-07 17:32:36,743_743:INFO:handle_variables:sftlf: loading LANDFRAC
2023-09-07 17:32:36,746_746:INFO:handle_variables:sftlf: loading axes
2023-09-07 17:32:36,747_747:INFO:handle_variables:sftlf: time 59.0 - 90.0
2023-09-07 17:32:36,759_759:INFO:handle_variables:sftlf: loading LANDFRAC
2023-09-07 17:32:36,762_762:INFO:handle_variables:sftlf: loading axes
.
.
.
2023-09-07 17:32:46,879_879:INFO:handle_variables:sftlf: time 15177.0 - 15208.0
2023-09-07 17:32:46,898_898:INFO:handle_variables:sftlf: loading LANDFRAC
2023-09-07 17:32:46,903_903:INFO:handle_variables:sftlf: loading axes
  File "/home/ac.zhang40/y/envs/e2c190/lib/python3.10/site-packages/e3sm_to_cmip/lib.py", line 116, in run_parallel
    out = res.result()
  File "/home/ac.zhang40/y/envs/e2c190/lib/python3.10/concurrent/futures/_base.py", line 458, in result
    return self.__get_result()
  File "/home/ac.zhang40/y/envs/e2c190/lib/python3.10/concurrent/futures/_base.py", line 403, in __get_result
    raise self._exception
Can't pickle <class '_cmor.CMORError'>: import of module '_cmor' failed
100%|████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:10<00:00, 10.63s/it]
2023-09-07 17:32:49,502_502:INFO:run_parallel:0 of 1 handlers complete
2023-09-07 17:32:49,502_502:ERROR:run_parallel:sftlf failed to complete
2023-09-07 17:32:49,502_502:ERROR:run_parallel:0 of 1 handlers complete

The root cause I think is that land fraction (cmip name: sftlf) is a fixed variable without time variation. Though E3SM output LANDFRAC as a time depended variable, and in e3sm_to_cmip, the handler is also coded as a regular variable ,i.e., with a time dimension. Possible solution is to separately generate a handler script for sftlf, following what has been done for PHIS (cmip name: orog), surface geopotential, which is also a fixed variable, but the E3SM output it with time dimension.

tomvothecoder commented 1 year ago

Hi @chengzhuzhang, I had a few notes and questions.

The root cause I think is that land fraction (cmip name: sftlf) is a fixed variable without time variation. Though E3SM output LANDFRAC as a time depended variable, and in e3sm_to_cmip, the handler is also coded as a regular variable ,i.e., with a time dimension.

If I'm understanding correctly, LANDFRAC has a time dimension but we need to CMORize to sftlf without a time dimension since this is a fixed variable without time variation?

Possible solution is to separately generate a handler script for sftlf, following what has been done for PHIS (cmip name: orog), surface geopotential, which is also a fixed variable, but the E3SM output it with time dimension.

I moved legacy handlers that I didn't refactor in PR #103 yet into cmor_handlers/vars. These handler modules don't use the generic handle_simple() and handle_variables() functions (maybe due to being fixed time variables) which makes them less straightforward to refactor.

We should probably avoid defining new handlers as Python modules.

Alternative solution

We might be able to update the generic handle_variables() method instead. We'll need some logic to detect fixed time dimension and CMORize without it.

chengzhuzhang commented 1 year ago

If I'm understanding correctly, LANDFRAC has a time dimension but we need to CMORize to sftlf without a time dimension since this is a fixed variable without time variation?

Yes! We should probably avoid defining new handlers as Python modules.

Alternative solution

We might be able to update the generic handle_variables() method instead. We'll need some logic to detect fixed time dimension and CMORize without it.

* https://github.com/E3SM-Project/e3sm_to_cmip/blob/c25eb2741d0e1096e7698b36af1edad152eb62c6/e3sm_to_cmip/cmor_handlers/handler.py#L155-L202

* https://github.com/E3SM-Project/e3sm_to_cmip/blob/c25eb2741d0e1096e7698b36af1edad152eb62c6/e3sm_to_cmip/lib.py#L439-L605

I agree it is better to try move away from legacy way and identify new python modules for handlers. Though the lib.py already has lots of logics to handle corner cases, and needs a refactor. I'm still challenged to make modification to lib.py at this point. I will plan to add another PR to add two regular variables. If it is possible, I hope they can be included in the e3sm-unified, because there are development in zppy ongoing to make cmorizing a standard task. Having both updates in e3sm_to_cmip to be included in unified will be really helpful..