wtbarnes / fiasco

Python interface to the CHIANTI atomic database
http://fiasco.readthedocs.io/
BSD 3-Clause "New" or "Revised" License
21 stars 16 forks source link

drparams footer is denoted differently in v9 #112

Closed wtbarnes closed 3 years ago

wtbarnes commented 3 years ago

Additionally, there appears to have been a change in the fe_10.drparams file format as the current drparams parser fails when trying to parse it with the following exception,

ValueError                                Traceback (most recent call last)
<ipython-input-4-e2ac303935f6> in <module>
----> 1 build_hdf5_dbase('/Users/willbarnes/Downloads/CHIANTI_9.0.1_data','/Users/willbarnes/Desktop/chianti-v9.0.1.h5',ask_before=False)

~/Documents/work/codes/fiasco/fiasco/util/setup_db.py in build_hdf5_dbase(ascii_dbase_root, hdf5_dbase_root, ask_before)
     76                 log.debug(f'Parsing file {af}')
     77                 parser = fiasco.io.Parser(af, ascii_dbase_root=ascii_dbase_root)
---> 78                 df = parser.parse()
     79                 if df is None:
     80                     log.warn(f'Not including {af} in {hdf5_dbase_root}')

~/Documents/work/codes/fiasco/fiasco/io/generic.py in parse(self)
     48                 break
     49             else:
---> 50                 self.preprocessor(table, line, i)
     51
     52         df = QTable(data=list(map(list, zip(*table))), names=self.headings)

~/Documents/work/codes/fiasco/fiasco/io/sources/ion_sources.py in preprocessor(self, table, line, index)
    294         else:
    295             if self._drparams_filetype == 1:
--> 296                 tmp = np.array(line[2:],dtype=float)
    297                 if index%2 == 0:
    298                     tmp_col = table[-1]

ValueError: could not convert string to float: 'Badnell,'

Originally posted by @wtbarnes in https://github.com/wtbarnes/fiasco/issues/50#issuecomment-592324983

wtbarnes commented 3 years ago

Copied from https://github.com/wtbarnes/fiasco/issues/50#issuecomment-592330065

Ok the reason these fail to parse is because the parser expects the footer, which contains the reference and other info about the data, to be fenced by -1 values, e.g. the fe_12.drparams file looks like,

    1
   26   12  1.3870e+03  1.0480e+04  3.9550e+04  1.4610e+05  4.0100e+05  7.2080e+05  0.0000e+00  0.0000e+00  0.0000e+00
   26   12  1.0740e-03  6.0800e-03  1.8870e-02  2.5400e-02  7.5800e-02  2.7730e-01  0.0000e+00  0.0000e+00  0.0000e+00
-1
%file:  fe_12.drparams
%parameters for dielectronic recombination rate coefficients
Determined from the coefficients listed at:
   http://amdpp.phys.strath.ac.uk/tamoc/DATA/RR/
These calculations have been performed by a collaboration of researchers at:
    Auburn University, Rollins College, and the University of Strathclyde.
 produced for the CHIANTI atomic database for astrophysical spectroscopy
 created by Ken Dere (GMU):  Wed Feb 15 11:13:10 2012
-1

but in the Fe X and XI drparams files, the initial -1 is missing so it tries to parse the footer as the data which obviously fails. It may be possible to just add an additional check for %file for denoting the footer.

wtbarnes commented 3 years ago

Copied from https://github.com/wtbarnes/fiasco/issues/50#issuecomment-592553151 by @kdere

things are a little different with the .drparams files. the number at the beginning of the file tell what type of data it expects and from this the format of the parameters is know. So, the -1 is not really necessary

kdere commented 3 years ago

Yes, in ChiantiPy it seems that the footer is not actually read. I fixed fe_10 and fe_11 and that will go out with the next release. Are there any others like this?

wtbarnes commented 3 years ago

I think I recall looking through all the files and these being the only cases where my existing readers failed, but I'm not entirely sure.

So is the plan in the future to still continue fencing the footers in the file with -1?