radis / radis

🌱 A fast line-by-line code for high-resolution infrared molecular spectra
http://radis.github.io/
GNU Lesser General Public License v3.0
211 stars 122 forks source link

CDSD-4000 loading problem & multitemperature fitting example unexpected behaviour #159

Closed BlehMaks closed 3 years ago

BlehMaks commented 3 years ago

Dear RADIS team,

As Mr. Pannier asked me, I am posting my problems here.

1) For this problem I used PyCharm 2020.2.3 and RADIS 0.9.25. I tried to use CDSD-4000 database for future fitting of non-equilibrium CO2 plasma spectra. As usage HITEMP database is more or less standard option for RADIS I decided to start from CDSD-4000. I have some issues with loading CDSD-4000 database file to RADIS's class SpectrumFactory for further calculations. As I understood, the best way to load the database is to create a configuration note for my database in .radis file and to use a built in to the class loader. But something goes wrong when I am trying to run it. To be more precise below is my configuration note for example for one of CDSD-4000 files:

Note in configuration file .radis [CDSD-4000-00500_00501] info = CDSD-4000, CO2, 4 main isotopes (12^C16^O_2, 13^C16^O_2, 12^C16^O18^O, 12^C16^O17^O) 00500_00501cm^-1 path = D:\Blekhshtein_phd\Files\Database_files\CDSD_4000\cdsd_00500_00501 format = cdsd-4000 parfuncfmt = cdsd levelsfmt = radis

And using this note RADIS throws and error:

Traceback for loading CDSD-4000 database problem Traceback (most recent call last): File "C:\Program Files\JetBrains\PyCharm 2020.2.3\plugins\python\helpers\pydev\_pydevd_bundle\pydevd_exec2.py", line 3, in Exec exec(exp, global_vars, local_vars) File "", line 1, in File "D:\Blekhshtein_phd\Files\Database_files\HAPI_HITRAN\venv\lib\site-packages\radis\lbl\loader.py", line 1172, in load_databank self._init_equilibrium_partition_functions(parfunc, parfuncfmt) File "D:\Blekhshtein_phd\Files\Database_files\HAPI_HITRAN\venv\lib\site-packages\radis\lbl\loader.py", line 1493, in _init_equilibrium_partition_functions parfunc, parfuncfmt, self.input.molecule, isotope=iso File "D:\Blekhshtein_phd\Files\Database_files\HAPI_HITRAN\venv\lib\site-packages\radis\lbl\loader.py", line 2219, in _build_partition_function_interpolator parsum = PartFuncCO2_CDSDtab(isotope, parfunc) File "D:\Blekhshtein_phd\Files\Database_files\HAPI_HITRAN\venv\lib\site-packages\radis\levels\partfunc_cdsd.py", line 87, in __init__ parsum = pd.read_csv(database, comment="#", delim_whitespace=True) File "D:\Blekhshtein_phd\Files\Database_files\HAPI_HITRAN\venv\lib\site-packages\pandas\io\parsers.py", line 688, in read_csv return _read(filepath_or_buffer, kwds) File "D:\Blekhshtein_phd\Files\Database_files\HAPI_HITRAN\venv\lib\site-packages\pandas\io\parsers.py", line 437, in _read filepath_or_buffer, encoding, compression File "D:\Blekhshtein_phd\Files\Database_files\HAPI_HITRAN\venv\lib\site-packages\pandas\io\common.py", line 243, in get_filepath_or_buffer raise ValueError(msg) **ValueError: Invalid file path or buffer object type: **

Mr. Pannier suggested an answer: "your config file does not give a path to the CDSD-4000 partition functions files. I think the error ("None") comes from this."

In this case how can I get partition functions for this database or calculate them? (Using HITRAN or HITEMP database there is no such problem, so partition functions for them are stored or calculated?)

2) I switched to my personal laptop where I have PyCharm 2020.3 and RADIS v0.9.26. I tried to run exactly validation case for multitemperature fit of Klarenaar 2017. But I got some problems with energy levels calculation which I have not experienced with older version of RADIS (0.9.25). Traceback of this example listed below:

Traceback for validation case problem D:\My_files\PHD_KIT\Python\Fitting_spectra\venv\lib\site-packages\radis\spectrum\spectrum.py:3459: UserWarning: Wavespace is not evenly spaced (0.000%) for transmittance_noslit. This may create problems when convolving with slit function + " This may create problems when convolving with slit function" Using database: HITEMP-CO2-TEST HITEMP-CO2-TEST ------- info : HITEMP-2010, CO2, 3 main isotope (CO2-626, 636, 628), 2283.7-2285.1 cm-1 path : ['D:\\My_files\\PHD_KIT\\Python\\Fitting_spectra\\venv\\lib\\site-packages\\radis\\test\\files\\cdsd_hitemp_09_fragment.txt'] format : cdsd-hitemp parfuncfmt : hapi levelsfmt : radis Calculating Non-Equilibrium Spectrum Physical Conditions ---------------------------------------- Tgas 491 K Trot 491 K Tvib 517,517,2641 K isotope 1,2 mole_fraction 0.06573632856818698 molecule CO2 path_length 10 cm pressure_mbar 20.0 mbar rot_distribution boltzmann self_absorption True state X vib_distribution treanor wavenum_max 2284.6000 cm-1 wavenum_min 2284.2000 cm-1 Computation Parameters ---------------------------------------- Tref 296 K broadening_max_width 1 cm-1 broadening_method fft cutoff 1e-25 cm-1/(#.cm-2) db_assumed_sorted True dbformat cdsd-hitemp dbpath D:/My_files/PHD_KIT/Python/Fitting_spectra/venv/lib/site-packages/radis/test/files/cdsd_hitemp_09_fr... dlm_log_pG 0.1375350788016573 dlm_log_pL 0.20180288881201608 include_neighbouring_lines True levelsfmt radis optimization min-RMS parfuncfmt hapi pseudo_continuum_threshold 0 wavenum_max_calc 2285.1010 cm-1 wavenum_min_calc 2283.7000 cm-1 waveunit cm-1 wstep 0.001 cm-1 Information ---------------------------------------- db_use_cached True export_lines True load_energies True lvl_use_cached True parallel False warning_broadening_threshold 0.01 warning_linestrength_cutoff 0.01 ---------------------------------------- Fetching Evib & Erot. If using this code several times you should consider updating the database directly. See functions in factory.py Fetching vib / rot energies for all 2819 transitions Fetched energies in 2s D:\My_files\PHD_KIT\Python\Fitting_spectra\venv\lib\site-packages\radis\misc\warning.py:241: NegativeEnergiesWarning: There are negative rotational energies in the database warnings.warn(WarningType(message)) Calculate weighted transition moment Calculated weighted transition moment in 0.09 Checked nonequilibrium parameters in 2.07s ... partition functions in 0.17s ... populations in 0.0s Calculated nonequilibrium populations (multiTvib) in 0.19s ... map partition functions in 0.01s ... corrected for populations and stimulated emission in 0.01s scaled nonequilibrium linestrength in 0.02s calculated emission integral in 0.00s Discarded 89.43% of lines (linestrength<1e-25cm-1/(#.cm-2)) Estimated error: 0.00%, Expected time saved: 0.3s Applied linestrength cutoff in 0.0s (expected time saved ~ 0.3s) Calculated lineshift in 0.00s Calculated broadening HWHM in 0.00s Calculating line broadening (298 lines: expect ~ 0.03s on 1 CPU) ... Precomputed DLM lineshapes (8) in 0.0s ... Initialized vectors in 0.0s ... Get closest matching line & fraction in 0.0s ... Distribute lines over DLM 0.0s ... Convolve and sum on spectral range 0.0s ... Initialized vectors in 0.0s ... Get closest matching line & fraction in 0.0s ... Distribute lines over DLM 0.0s ... Convolve and sum on spectral range 0.0s Calculated line broadening in 0.01s Calculated other spectral quantities in 0.01s Spectrum calculated in 2.33s (before object generation) Generated Spectrum object in 0.00s Spectrum calculated in 2.33s D:\My_files\PHD_KIT\Python\Fitting_spectra\venv\lib\site-packages\radis\misc\warning.py:241: NegativeEnergiesWarning: There are negative rotational energies in the database warnings.warn(WarningType(message)) Traceback (most recent call last): File "D:\My_files\PHD_KIT\Python\Fitting_spectra\venv\lib\site-packages\astropy\units\format\generic.py", line 566, in _do_parse return cls._parse_unit(s, detailed_exception=False) File "D:\My_files\PHD_KIT\Python\Fitting_spectra\venv\lib\site-packages\astropy\units\format\generic.py", line 487, in _parse_unit raise ValueError() ValueError During handling of the above exception, another exception occurred: Traceback (most recent call last): File "D:\My_files\PHD_KIT\Python\Fitting_spectra\venv\lib\site-packages\astropy\units\format\generic.py", line 459, in _get_unit return cls._parse_unit(t.value) File "D:\My_files\PHD_KIT\Python\Fitting_spectra\venv\lib\site-packages\astropy\units\format\generic.py", line 485, in _parse_unit s, did_you_mean(s, registry))) ValueError: I is not a valid unit. During handling of the above exception, another exception occurred: Traceback (most recent call last): File "D:\My_files\PHD_KIT\Python\Fitting_spectra\venv\lib\site-packages\astropy\units\core.py", line 1854, in __call__ return f.parse(s) File "D:\My_files\PHD_KIT\Python\Fitting_spectra\venv\lib\site-packages\astropy\units\format\generic.py", line 551, in parse result = cls._do_parse(s, debug=debug) File "D:\My_files\PHD_KIT\Python\Fitting_spectra\venv\lib\site-packages\astropy\units\format\generic.py", line 569, in _do_parse return cls._parser.parse(s, lexer=cls._lexer, debug=debug) File "D:\My_files\PHD_KIT\Python\Fitting_spectra\venv\lib\site-packages\astropy\extern\ply\yacc.py", line 333, in parse return self.parseopt_notrack(input, lexer, debug, tracking, tokenfunc) File "D:\My_files\PHD_KIT\Python\Fitting_spectra\venv\lib\site-packages\astropy\extern\ply\yacc.py", line 1063, in parseopt_notrack lookahead = get_token() # Get the next token File "D:\My_files\PHD_KIT\Python\Fitting_spectra\venv\lib\site-packages\astropy\extern\ply\lex.py", line 350, in token newtok = func(tok) File "D:\My_files\PHD_KIT\Python\Fitting_spectra\venv\lib\site-packages\astropy\units\format\generic.py", line 165, in t_UNIT t.value = cls._get_unit(t) File "D:\My_files\PHD_KIT\Python\Fitting_spectra\venv\lib\site-packages\astropy\units\format\generic.py", line 463, in _get_unit t.lexpos, str(e))) ValueError: At col 0, I is not a valid unit. During handling of the above exception, another exception occurred: Traceback (most recent call last): File "D:\My_files\PHD_KIT\Python\Fitting_spectra\venv\lib\site-packages\radis\phys\units.py", line 81, in conv2 a = quantity * Unit(fromunit) File "D:\My_files\PHD_KIT\Python\Fitting_spectra\venv\lib\site-packages\radis\phys\units.py", line 45, in Unit unit = u.Unit(st, *args, **kwargs) File "D:\My_files\PHD_KIT\Python\Fitting_spectra\venv\lib\site-packages\astropy\units\core.py", line 1874, in __call__ raise ValueError(msg) ValueError: 'I/I0' did not parse as unit: At col 0, I is not a valid unit. If this is meant to be a custom unit, define it with 'u.def_unit'. To have it recognized inside a file reader or other code, enable it with 'u.add_enabled_units'. For details, see https://docs.astropy.org/en/latest/units/combining_and_defining.html

Could someone please tell me what am I doing wrong?

Thank you in advance for your answers!

Best regards, Marry Christmas and Happy new year, Maksim

erwanp commented 3 years ago

Hello @BlehMaks and merry Xmas ;)

The CDSD partition_functions.txt can be downloaded from the CDSD-4000 FTP : ftp://ftp.iao.ru/pub/CDSD-4000/ . I improved the docs in 618b41a to give the path and help future users !

By default (HITRAN/HITEMP) RADIS uses partition functions tabulated with TIPS 2011 (now upgraded to TIPS 2017 #152 ), and retrieved with HAPI.

If you plan to calculate under non-equilibrium conditions, things gets more complicated : tabulated nonequilibrium partition functions are not available, therefore RADIS computes them on the fly, but it is also slower. There is Feature Request to tabulated and cache #45 but it is not implemented yet !

For nonequilibrium partition functions you'll need rovibrational energies :


  1. I ran the Klarenaar test case using 0.9.26 and did not get the error you're having. Let's discuss this on Slack !
erwanp commented 3 years ago

For 2. , as written on Slack :

your test file was not from radis but from the radis-examples project, which has no automatic testing. So a small change in user syntax created that bug that remained undetected (probably since #106 ) : we now define transmittance units as adimensionned (unit="") rather than unit="I/I0"

Fixed in https://github.com/radis/radis-examples/commit/468424324ddd2b95a22dcc077aa00d61b38e09ed