search

E3SM-Project / e3sm_to_cmip

Tools to CMORize E3SM output
https://e3sm-to-cmip.readthedocs.io/en/latest/
MIT License
7 stars 7 forks source link

Fix segmentation fault due to early `cmor.close()` call #234

Closed tomvothecoder closed 11 months ago

tomvothecoder commented 11 months ago

Log output after fix:

2023-12-18 21:48:54,624 [INFO]: __main__.py(__init__:142) >> --------------------------------------
2023-12-18 21:48:54,624 [INFO]: __main__.py(__init__:142) >> --------------------------------------
2023-12-18 21:48:54,624_624:INFO:__init__:--------------------------------------
2023-12-18 21:48:54,626 [INFO]: __main__.py(__init__:143) >> | E3SM to CMIP Configuration
2023-12-18 21:48:54,626 [INFO]: __main__.py(__init__:143) >> | E3SM to CMIP Configuration
2023-12-18 21:48:54,626_626:INFO:__init__:| E3SM to CMIP Configuration
2023-12-18 21:48:54,627 [INFO]: __main__.py(__init__:144) >> --------------------------------------
2023-12-18 21:48:54,627 [INFO]: __main__.py(__init__:144) >> --------------------------------------
2023-12-18 21:48:54,627_627:INFO:__init__:--------------------------------------
2023-12-18 21:48:54,628 [INFO]: __main__.py(__init__:145) >>     * var_list='['pr', 'tas', 'rsds', 'rlds', 'rsus']'
2023-12-18 21:48:54,628 [INFO]: __main__.py(__init__:145) >>     * var_list='['pr', 'tas', 'rsds', 'rlds', 'rsus']'
2023-12-18 21:48:54,628_628:INFO:__init__:    * var_list='['pr', 'tas', 'rsds', 'rlds', 'rsus']'
2023-12-18 21:48:54,629 [INFO]: __main__.py(__init__:146) >>     * input_path='/lcrc/group/e3sm/ac.forsyth2/zppy_test_complete_run_output/test_unified_1.9.2rc1/v2.LR.historical_0201/post/atm/180x360_aave/ts/monthly/2yr'
2023-12-18 21:48:54,629 [INFO]: __main__.py(__init__:146) >>     * input_path='/lcrc/group/e3sm/ac.forsyth2/zppy_test_complete_run_output/test_unified_1.9.2rc1/v2.LR.historical_0201/post/atm/180x360_aave/ts/monthly/2yr'
2023-12-18 21:48:54,629_629:INFO:__init__:    * input_path='/lcrc/group/e3sm/ac.forsyth2/zppy_test_complete_run_output/test_unified_1.9.2rc1/v2.LR.historical_0201/post/atm/180x360_aave/ts/monthly/2yr'
2023-12-18 21:48:54,630 [INFO]: __main__.py(__init__:147) >>     * output_path='/home/ac.tvo/E3SM-Project/e3sm_to_cmip/qa/12-18-23-zppy-seg'
2023-12-18 21:48:54,630 [INFO]: __main__.py(__init__:147) >>     * output_path='/home/ac.tvo/E3SM-Project/e3sm_to_cmip/qa/12-18-23-zppy-seg'
2023-12-18 21:48:54,630_630:INFO:__init__:    * output_path='/home/ac.tvo/E3SM-Project/e3sm_to_cmip/qa/12-18-23-zppy-seg'
2023-12-18 21:48:54,631 [INFO]: __main__.py(__init__:148) >>     * precheck_path='None'
2023-12-18 21:48:54,631 [INFO]: __main__.py(__init__:148) >>     * precheck_path='None'
2023-12-18 21:48:54,631_631:INFO:__init__:    * precheck_path='None'
2023-12-18 21:48:54,631 [INFO]: __main__.py(__init__:149) >>     * freq='mon'
2023-12-18 21:48:54,631 [INFO]: __main__.py(__init__:149) >>     * freq='mon'
2023-12-18 21:48:54,631_631:INFO:__init__:    * freq='mon'
2023-12-18 21:48:54,632 [INFO]: __main__.py(__init__:150) >>     * realm='atm'
2023-12-18 21:48:54,632 [INFO]: __main__.py(__init__:150) >>     * realm='atm'
2023-12-18 21:48:54,632_632:INFO:__init__:    * realm='atm'
2023-12-18 21:48:54,633 [INFO]: __main__.py(__init__:151) >>     * Writing log output file to: logs/20231218_214854_621672
2023-12-18 21:48:54,633 [INFO]: __main__.py(__init__:151) >>     * Writing log output file to: logs/20231218_214854_621672
2023-12-18 21:48:54,633_633:INFO:__init__:    * Writing log output file to: logs/20231218_214854_621672
2023-12-18 21:48:57,401 [WARNING]: utils.py(derive_handlers:218) >> No handlers could be derived for the variables: ['rsds', 'rlds', 'rsus']. Make sure the input E3SM datasets have the variables needed derivation.
2023-12-18 21:48:57,401 [WARNING]: utils.py(derive_handlers:218) >> No handlers could be derived for the variables: ['rsds', 'rlds', 'rsus']. Make sure the input E3SM datasets have the variables needed derivation.
2023-12-18 21:48:57,401_401:WARNING:derive_handlers:No handlers could be derived for the variables: ['rsds', 'rlds', 'rsus']. Make sure the input E3SM datasets have the variables needed derivation.
2023-12-18 21:48:57,403 [INFO]: __main__.py(_get_handlers:220) >> --------------------------------------
2023-12-18 21:48:57,403 [INFO]: __main__.py(_get_handlers:220) >> --------------------------------------
2023-12-18 21:48:57,403_403:INFO:_get_handlers:--------------------------------------
2023-12-18 21:48:57,403 [INFO]: __main__.py(_get_handlers:221) >> | Derived CMIP6 Variable Handlers
2023-12-18 21:48:57,403 [INFO]: __main__.py(_get_handlers:221) >> | Derived CMIP6 Variable Handlers
2023-12-18 21:48:57,403_403:INFO:_get_handlers:| Derived CMIP6 Variable Handlers
2023-12-18 21:48:57,404 [INFO]: __main__.py(_get_handlers:222) >> --------------------------------------
2023-12-18 21:48:57,404 [INFO]: __main__.py(_get_handlers:222) >> --------------------------------------
2023-12-18 21:48:57,404_404:INFO:_get_handlers:--------------------------------------
2023-12-18 21:48:57,405 [INFO]: __main__.py(_get_handlers:224) >>     * 'pr' -> ['PRECC', 'PRECL']
2023-12-18 21:48:57,405 [INFO]: __main__.py(_get_handlers:224) >>     * 'pr' -> ['PRECC', 'PRECL']
2023-12-18 21:48:57,405_405:INFO:_get_handlers:    * 'pr' -> ['PRECC', 'PRECL']
2023-12-18 21:48:57,406 [INFO]: __main__.py(_get_handlers:224) >>     * 'tas' -> ['TREFHT']
2023-12-18 21:48:57,406 [INFO]: __main__.py(_get_handlers:224) >>     * 'tas' -> ['TREFHT']
2023-12-18 21:48:57,406_406:INFO:_get_handlers:    * 'tas' -> ['TREFHT']
2023-12-18 21:48:57,431 [INFO]: __main__.py(_run:737) >> --------------------------------------
2023-12-18 21:48:57,431 [INFO]: __main__.py(_run:737) >> --------------------------------------
2023-12-18 21:48:57,431_431:INFO:_run:--------------------------------------
2023-12-18 21:48:57,432 [INFO]: __main__.py(_run:738) >> | Running E3SM to CMIP in Serial
2023-12-18 21:48:57,432 [INFO]: __main__.py(_run:738) >> | Running E3SM to CMIP in Serial
2023-12-18 21:48:57,432_432:INFO:_run:| Running E3SM to CMIP in Serial
2023-12-18 21:48:57,433 [INFO]: __main__.py(_run:739) >> --------------------------------------
2023-12-18 21:48:57,433 [INFO]: __main__.py(_run:739) >> --------------------------------------
2023-12-18 21:48:57,433_433:INFO:_run:--------------------------------------
2023-12-18 21:48:57,435 [INFO]: __main__.py(_run_serial:796) >> Trying to CMORize with handler: {'name': 'pr', 'units': 'kg m-2 s-1', 'raw_variables': ['PRECC', 'PRECL'], 'table': 'CMIP6_Amon.json', 'unit_conversion': None, 'formula': '(PRECC + PRECL) * 1000.0', 'formula_method': <function pr at 0x1550ac6d9760>, 'positive': None, 'levels': None, 'output_data': None, 'method': <bound method VarHandler.cmorize of <e3sm_to_cmip.cmor_handlers.handler.VarHandler object at 0x1550ac745590>>}
2023-12-18 21:48:57,435 [INFO]: __main__.py(_run_serial:796) >> Trying to CMORize with handler: {'name': 'pr', 'units': 'kg m-2 s-1', 'raw_variables': ['PRECC', 'PRECL'], 'table': 'CMIP6_Amon.json', 'unit_conversion': None, 'formula': '(PRECC + PRECL) * 1000.0', 'formula_method': <function pr at 0x1550ac6d9760>, 'positive': None, 'levels': None, 'output_data': None, 'method': <bound method VarHandler.cmorize of <e3sm_to_cmip.cmor_handlers.handler.VarHandler object at 0x1550ac745590>>}
2023-12-18 21:48:57,435_435:INFO:_run_serial:Trying to CMORize with handler: {'name': 'pr', 'units': 'kg m-2 s-1', 'raw_variables': ['PRECC', 'PRECL'], 'table': 'CMIP6_Amon.json', 'unit_conversion': None, 'formula': '(PRECC + PRECL) * 1000.0', 'formula_method': <function pr at 0x1550ac6d9760>, 'positive': None, 'levels': None, 'output_data': None, 'method': <bound method VarHandler.cmorize of <e3sm_to_cmip.cmor_handlers.handler.VarHandler object at 0x1550ac745590>>}
2023-12-18 21:48:57,435 [INFO]: handler.py(cmorize:209) >> pr: Starting CMORizing
2023-12-18 21:48:57,435 [INFO]: handler.py(cmorize:209) >> pr: Starting CMORizing
2023-12-18 21:48:57,435_435:INFO:cmorize:pr: Starting CMORizing
2023-12-18 21:48:57,628 [INFO]: handler.py(_setup_cmor_module:333) >> pr: CMOR setup complete
2023-12-18 21:48:57,628 [INFO]: handler.py(_setup_cmor_module:333) >> pr: CMOR setup complete
2023-12-18 21:48:57,628_628:INFO:_setup_cmor_module:pr: CMOR setup complete
2023-12-18 21:48:57,630 [INFO]: handler.py(cmorize:239) >> pr: loading E3SM variables dict_keys(['PRECC', 'PRECL'])
2023-12-18 21:48:57,630 [INFO]: handler.py(cmorize:239) >> pr: loading E3SM variables dict_keys(['PRECC', 'PRECL'])
2023-12-18 21:48:57,630_630:INFO:cmorize:pr: loading E3SM variables dict_keys(['PRECC', 'PRECL'])
2023-12-18 21:48:59,886 [INFO]: handler.py(cmorize:247) >> pr: creating CMOR variable with CMOR axis objects.
2023-12-18 21:48:59,886 [INFO]: handler.py(cmorize:247) >> pr: creating CMOR variable with CMOR axis objects.
2023-12-18 21:48:59,886_886:INFO:cmorize:pr: creating CMOR variable with CMOR axis objects.
2023-12-18 21:48:59,907 [INFO]: handler.py(_cmor_write_with_time:676) >> pr: time span 0.0 - 730.0
2023-12-18 21:48:59,907 [INFO]: handler.py(_cmor_write_with_time:676) >> pr: time span 0.0 - 730.0
2023-12-18 21:48:59,907_907:INFO:_cmor_write_with_time:pr: time span 0.0 - 730.0
2023-12-18 21:48:59,908 [INFO]: handler.py(_cmor_write_with_time:680) >> pr: Writing variable to file...
2023-12-18 21:48:59,908 [INFO]: handler.py(_cmor_write_with_time:680) >> pr: Writing variable to file...
2023-12-18 21:48:59,908_908:INFO:_cmor_write_with_time:pr: Writing variable to file...
2023-12-18 21:48:59,970 [INFO]: handler.py(cmorize:239) >> pr: loading E3SM variables dict_keys(['PRECC', 'PRECL'])
2023-12-18 21:48:59,970 [INFO]: handler.py(cmorize:239) >> pr: loading E3SM variables dict_keys(['PRECC', 'PRECL'])
2023-12-18 21:48:59,970_970:INFO:cmorize:pr: loading E3SM variables dict_keys(['PRECC', 'PRECL'])
2023-12-18 21:49:00,029 [INFO]: handler.py(cmorize:247) >> pr: creating CMOR variable with CMOR axis objects.
2023-12-18 21:49:00,029 [INFO]: handler.py(cmorize:247) >> pr: creating CMOR variable with CMOR axis objects.
2023-12-18 21:49:00,029_029:INFO:cmorize:pr: creating CMOR variable with CMOR axis objects.
2023-12-18 21:49:00,049 [INFO]: handler.py(_cmor_write_with_time:676) >> pr: time span 730.0 - 1460.0
2023-12-18 21:49:00,049 [INFO]: handler.py(_cmor_write_with_time:676) >> pr: time span 730.0 - 1460.0
2023-12-18 21:49:00,049_049:INFO:_cmor_write_with_time:pr: time span 730.0 - 1460.0
2023-12-18 21:49:00,050 [INFO]: handler.py(_cmor_write_with_time:680) >> pr: Writing variable to file...
2023-12-18 21:49:00,050 [INFO]: handler.py(_cmor_write_with_time:680) >> pr: Writing variable to file...
2023-12-18 21:49:00,050_050:INFO:_cmor_write_with_time:pr: Writing variable to file...
2023-12-18 21:49:00,502 [INFO]: __main__.py(_run_serial:821) >> Finished pr, 1/2 jobs complete
2023-12-18 21:49:00,502 [INFO]: __main__.py(_run_serial:821) >> Finished pr, 1/2 jobs complete
2023-12-18 21:49:00,502_502:INFO:_run_serial:Finished pr, 1/2 jobs complete
2023-12-18 21:49:00,504 [INFO]: __main__.py(_run_serial:796) >> Trying to CMORize with handler: {'name': 'tas', 'units': 'K', 'raw_variables': ['TREFHT'], 'table': 'CMIP6_Amon.json', 'unit_conversion': None, 'formula': None, 'positive': None, 'levels': None, 'output_data': None, 'method': <bound method VarHandler.cmorize of <e3sm_to_cmip.cmor_handlers.handler.VarHandler object at 0x1550ac747f50>>}
2023-12-18 21:49:00,504 [INFO]: __main__.py(_run_serial:796) >> Trying to CMORize with handler: {'name': 'tas', 'units': 'K', 'raw_variables': ['TREFHT'], 'table': 'CMIP6_Amon.json', 'unit_conversion': None, 'formula': None, 'positive': None, 'levels': None, 'output_data': None, 'method': <bound method VarHandler.cmorize of <e3sm_to_cmip.cmor_handlers.handler.VarHandler object at 0x1550ac747f50>>}
...
2023-12-18 21:49:01,047_047:INFO:_run_serial:Finished tas, 2/2 jobs complete
2023-12-18 21:49:01,048 [INFO]: __main__.py(_run_serial:836) >> 2 of 2 handlers complete
2023-12-18 21:49:01,048 [INFO]: __main__.py(_run_serial:836) >> 2 of 2 handlers complete
2023-12-18 21:49:01,048_048:INFO:_run_serial:2 of 2 handlers complete