Compatibility with CHIANTI v9

[wtbarnes]

CHIANTI v9 has now been released (see Dere et al., 2019) with additional modifications for how the level populations are calculated and a new data format.

Need to look into specifically what modifications need to be made to the level populations as well as the readers.

EDIT: This is really a "meta-issue" for general incompatibility. Try to create individual issues for each area of incompatibility.

[wtbarnes]

There are at least two new file formats: .rrlvl and .auto. Need to find out more about how these are used and how to parse them.

[wtbarnes]

Additionally, there appears to have been a change in the fe_10.drparams file format as the current drparams parser fails when trying to parse it with the following exception,

ValueError                                Traceback (most recent call last)
<ipython-input-4-e2ac303935f6> in <module>
----> 1 build_hdf5_dbase('/Users/willbarnes/Downloads/CHIANTI_9.0.1_data','/Users/willbarnes/Desktop/chianti-v9.0.1.h5',ask_before=False)

~/Documents/work/codes/fiasco/fiasco/util/setup_db.py in build_hdf5_dbase(ascii_dbase_root, hdf5_dbase_root, ask_before)
     76                 log.debug(f'Parsing file {af}')
     77                 parser = fiasco.io.Parser(af, ascii_dbase_root=ascii_dbase_root)
---> 78                 df = parser.parse()
     79                 if df is None:
     80                     log.warn(f'Not including {af} in {hdf5_dbase_root}')

~/Documents/work/codes/fiasco/fiasco/io/generic.py in parse(self)
     48                 break
     49             else:
---> 50                 self.preprocessor(table, line, i)
     51
     52         df = QTable(data=list(map(list, zip(*table))), names=self.headings)

~/Documents/work/codes/fiasco/fiasco/io/sources/ion_sources.py in preprocessor(self, table, line, index)
    294         else:
    295             if self._drparams_filetype == 1:
--> 296                 tmp = np.array(line[2:],dtype=float)
    297                 if index%2 == 0:
    298                     tmp_col = table[-1]

ValueError: could not convert string to float: 'Badnell,'

[wtbarnes]

ping @fluxtransport

[wtbarnes]

Ok the reason these fail to parse is because the parser expects the footer, which contains the reference and other info about the data, to be fenced by -1 values, e.g. the fe_12.drparams file looks like,

    1
   26   12  1.3870e+03  1.0480e+04  3.9550e+04  1.4610e+05  4.0100e+05  7.2080e+05  0.0000e+00  0.0000e+00  0.0000e+00
   26   12  1.0740e-03  6.0800e-03  1.8870e-02  2.5400e-02  7.5800e-02  2.7730e-01  0.0000e+00  0.0000e+00  0.0000e+00
-1
%file:  fe_12.drparams
%parameters for dielectronic recombination rate coefficients
Determined from the coefficients listed at:
   http://amdpp.phys.strath.ac.uk/tamoc/DATA/RR/
These calculations have been performed by a collaboration of researchers at:
    Auburn University, Rollins College, and the University of Strathclyde.
 produced for the CHIANTI atomic database for astrophysical spectroscopy
 created by Ken Dere (GMU):  Wed Feb 15 11:13:10 2012
-1

but in the Fe X and XI drparams files, the initial -1 is missing so it tries to parse the footer as the data which obviously fails. It may be possible to just add an additional check for %file for denoting the footer.

[wtbarnes]

This posting on the CHIANTI mailing list may be of some help.

[kdere]

things are a little different with the .drparams files. the number at the beginning of the file tell what type of data it expects and from this the format of the parameters is know. So, the -1 is not really necessary

[wtbarnes]

Fair point. I was using the -1 as a general marker across all file types to denote the footer.

[wtbarnes]

See #48

[wtbarnes]

This posting better enumerates the needed changes: https://groups.google.com/g/chianti/c/hzJyjhkAhAw