num_energy_groups changes rapidly with time

[grazwegian]

When reviewing a issue related to the ELUT I noticed that the reported value in num_energy_groups seems to be different for each energy bin ranging between 3 and 15, I would have expected a constant value of 18 the total number of energy bins in for each data point. The DPDD gives the description "number of energy bands" which would seem to match my expectation. Checking with fv gives the same result suggesting this is not related to the IDL software.

e.g. using the file from the imaging demo: fits_path = solo_L1_stix-sci-xray-cpd_20200607T213709-20200607T215208_V01_1178428688-49155.fits data = stx_read_fits(fits_path, 'data') data.num_energy_groups = [15 12 9 6 3 15 12 9 6 3 15 12 9 6 3 15 12 9 6 3 15 12 9 6 3 15 12 9 6 3 15 12 9 6 3 15 12 9 6 3 15 12 9 6 3]

[samaloney]

Yep I see the same from a quick look at the source code all we do is extract the value from the the packet so don't think it on the fits side need to check

• [x] the raw packet values
• [x] and then the IDB version.

https://github.com/i4Ds/STIXCore/blob/f628d3e7892542a33745d23b571962bd4d94f9fb/stixcore/products/level0/scienceL0.py#L350-L352

I should note that I expect the value to change and represent the number of channels in the current science sub structure which may vary to fit inside max packet size.

Code to reproduce

from astropy.io import fits

hdul = fits.open('http://pub099.cs.technik.fhnw.ch/data/fits/L1/2020/06/07/SCI/solo_L1_stix-sci-xray-cpd_20200607T213709-20200607T215208_V01_1178428688-49155.fits')

hdul[2].data['num_energy_groups']
# array([15, 12,  9,  6,  3, 15, 12,  9,  6,  3, 15, 12,  9,  6,  3, 15, 12,
#        9,  6,  3, 15, 12,  9,  6,  3, 15, 12,  9,  6,  3, 15, 12,  9,  6,
#        3, 15, 12,  9,  6,  3, 15, 12,  9,  6,  3], dtype=uint8)

hdul[2].data['num_energy_groups'].sum()
# 405

hdul[1].data['energy_bin_edge_mask'].sum()
# 19

hdul[1].data['energy_bin_edge_mask']
# array([[0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0,
        0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]], dtype=uint8)

I'd expect the num_energy_groups / n_energy_bin == n_times but this doesn't hold

[samaloney]

Ok yes the is an issue, the raw parameter look like this

packets.get('NIX00258')
[Parameter(name=NIX00258, value=[10], children=None, order=60),
 Parameter(name=NIX00258, value=[6, 4], children=None, order=60),
 Parameter(name=NIX00258, value=[10], children=None, order=60),
 Parameter(name=NIX00258, value=[2, 8], children=None, order=60),
 Parameter(name=NIX00258, value=[8, 2], children=None, order=60),
 Parameter(name=NIX00258, value=[10], children=None, order=60),
 Parameter(name=NIX00258, value=[4, 6], children=None, order=60)
 ...
 ]

These are then turned into a flat array e.g. [10, 6, 4, 10, 2, 8, 8,...] this flat array is later sliced by the unique time indices this will not match properly. What should probably happen is each list of numbers should be summed e.g. [10, 10, 10, 10, ....] and directly assigned to the final data structure.

[samaloney]

In the end this repeater doesn't really map back to anything meaningful in the science data. It required to define and parse the TM but it doesn't correspond to anything. The only use is as a check because sum(num_energy_groups) = n_time x n_energies. I'm going to suggest we remove it completely.

[nicHoch]

👍 for removing it @grazwegian for what purpose you were looking into the "num_energy_groups" value?

[grazwegian]

It was while I was looking into this issue: https://github.com/i4Ds/STIX-GSW/pull/185, I was trying to see if we had enough information in the data structure/extension to specify the energy binning. In the end we are now using the energy bin mask.