Error with app-yoda2h5 | Paramater datatype

[mfieg]

Hello - Thank you in advance for your attention and advice with my issue. I am having a problem with the app-yoda2h5 and app-build commands. I have used app-build with a collaborator's data, and it works fine. When I use my own data via pythia8.2, app-yoda2h5 runs successfully, but then when I use app-build I get an error. The returned error after running app-build mc.hdf5 --order 2,2

Traceback (most recent call last):
  File "/home/maxfieg/.local/bin/app-build", line 56, in <module>
    DATA, binids, pnames, rankIdx, xmin, xmax = app.io.readInputDataH5(args[0], opts.WEIGHTS)
  File "/home/maxfieg/.local/lib/python3.6/site-packages/apprentice/io.py", line 14, in readInputDataH5
    pnames      = app.io.readPnamesH5(fname, xfield="params")
  File "/home/maxfieg/.local/lib/python3.6/site-packages/apprentice/io.py", line 358, in readPnamesH5
    pnames = [p.astype(str) for p in f.get(xfield).attrs["names"]]
  File "/home/maxfieg/.local/lib/python3.6/site-packages/apprentice/io.py", line 358, in <listcomp>
    pnames = [p.astype(str) for p in f.get(xfield).attrs["names"]]
AttributeError: 'str' object has no attribute 'astype'

It seems like in the prior step when I ran app-yoda2h5 yodas , the parameter names are being stored improperly, but I cannot find why this is happening. I have attached my yodas directory which is a directory containing my yoda files for 2 runs, as well as the relevant params.dat file for each run; I've also attached the mc.hdf5 file that app-yoda2h5 returns as mc.zip. The pythia command card that I am using is also in the respective folder. (Note that I have also tried this for 20 runs but am merely sending 2 here for brevity.)

yodas.zip mc.zip

To Reproduce Steps to reproduce the behavior:

1. app-yoda2h5 yodas --pname params.dat
2. app-build mc.hdf5 --order 2,2

Expected behavior I expected to have an output json file which I can use to predict my histograms based on an input params.dat file.

Desktop (please complete the following information):

• OS: Ubuntu 18.04
• Browser FireFox
• Python3.6
• Rivet 3.1.2
• Pythia8.2
  • with HepMC 2.06

Additional context I have used h5py to look closer at my collegeue's (succesful) h5 file, and my own (problematic) h5 file. Based on the TraceBack copied above, when I run the line in question, Line358 in io.py under the function readPnameH5. I notice a difference in the parameter data type (S11 Vs object). i.e. For my collegue's succeful h5 file:

>>>f=h5py.File("newscan.h5","r")
>>> f.get("params").attrs["names"]
array([b'lowMEnhance', b'SaSepsilon', b'maxAXB', b'maxAX', b'mMin',
       b'mResMax', b'maxXB', b'maxXX', b'sigma'], dtype='|S11')

For my Problematic h5 file:

>>> f=h5py.File(mc.hdf5,"r")
>>> f.get("params").attrs["names"]
array(['maxXB', 'maxXX'], dtype=object)

I am not sure this is helpful but this is the only real difference I can find between our h5 files, and it probably has something to do with the error.

I have also tried creating the h5 file using my college's yoda files, and the same result occurs.
Thanks Again

[mfieg]

Alright I did find a workaround for the meantime - in io.py, I changed line 358 from p for p.astype(str) in f.get(xfield).attrs["names"] to p for p in f.get(xfield).attrs["names"] Wondering if this will become problematic for other functions, but for now app-build seems to work. I was able to follow though with the rest of the minimization procedure via app-tune2.

[iamholger]

Thanks @mfieg . We will make this more robust.

[smrenna]

This is the same issue I am seeing > 1 year later, and I made a similar fix. @mfieg did your results look sensible in the end after this change?

[smrenna]

@mfieg: Did app-build mc.hdf5 --order 2,2 give you a sensible result?

[mfieg]

@smrenna Yes I think so. While I can't remember what checks I did on this hotfix, I have been consistently using apprentice for the past year with sensible results.

[smrenna]

@mfieg : thank you for the quick reply. Did the "predictions" yoda file produced by your fit also make sense? I have a case from a user where the fit seems to make sense, but the "predictions" yoda file comes out with different binnings, etc.

[mfieg]

@smrenna Yes my fit makes sense. I've never seen anything like that - where the predictions.yoda file comes out with different binnings.