pymcpltool --stats errors with universal weight files with >1e4 particles

[tkittel]

I am not sure when this was introduced, but for instance the file at:

http://tkittel.web.cern.ch/tkittel/mcplissue49.mcpl.gz

Gives an error when combining the last block with the others:

$> pymcpltool --stats mcplissue49.mcpl.gz
Traceback (most recent call last):
  File "/home/thki/dgdepfixer_install/venv/bin/pymcpltool", line 10, in <module>
    sys.exit(main())
  File "/home/thki/dgdepfixer_install/venv/lib64/python3.5/site-packages/mcpl.py", line 1641, in main
    app_pymcpltool()
  File "/home/thki/dgdepfixer_install/venv/lib64/python3.5/site-packages/mcpl.py", line 1141, in app_pymcpltool
    dump_stats(f)
  File "/home/thki/dgdepfixer_install/venv/lib64/python3.5/site-packages/mcpl.py", line 1472, in dump_stats
    stats = collect_stats(stats,bin_data=False)
  File "/home/thki/dgdepfixer_install/venv/lib64/python3.5/site-packages/mcpl.py", line 1350, in collect_stats
    sc.add_data(getattr(pb,s),vals_weight)
  File "/home/thki/dgdepfixer_install/venv/lib64/python3.5/site-packages/mcpl.py", line 1260, in add_data
    new_sumwx = a.sum() if w is None else (a*w).sum()
ValueError: operands could not be broadcast together with shapes (17,) (10000,) 

(note: the default particle block size is 10000 and this file has 10017 particles)

[tkittel]

So this only affects files with universal weights. Consider the two files, differing only in whether or not they use universal weights (uw):

• http://tkittel.web.cern.ch/tkittel/mcplissue49_uw.mcpl.gz
• http://tkittel.web.cern.ch/tkittel/mcplissue49_nouw.mcpl.gz Running a small test script:

  import mcpl
  for fn in ('mcplissue49_uw.mcpl.gz','mcplissue49_nouw.mcpl.gz'):
  with mcpl.MCPLFile(fn) as f:
      print('Opened %s with %i particles and blocklength %i'%(fn,f.nparticles,f.blocklength))
      for i,p in enumerate(f.particle_blocks):
          print('   block#%i: len(p.x)=%i, len(p.weight)=%i'%(i,len(p.x),len(p.weight)))

  yields:

  Opened mcplissue49_uw.mcpl.gz with 10900 particles and blocklength 10000
  block#0: len(p.x)=10000, len(p.weight)=10000
  block#1: len(p.x)=900, len(p.weight)=10000
  Opened mcplissue49_nouw.mcpl.gz with 10900 particles and blocklength 10000
  block#0: len(p.x)=10000, len(p.weight)=10000
  block#1: len(p.x)=900, len(p.weight)=900

  The bug is that len(p.weight) in block 1 of the _uw file is 10000 instead of 900.

[tkittel]

I figured out a fix, I guess we need a 1.3.1 release in the near future.

[tkittel]

The issue actually concerns all variables not stored on-disk in a given file, but inferred from meta-data (universal pdgcode, polarisation, ...).

[tkittel]

I am assuming this was not discovered before since my testing now used a newer Numpy version than previously. Probably Numpy was previously more tolerant of multiplying arrays of unequal length.

[tkittel]

Fixed in MCPL 1.3.1.