search

i4Ds / STIXCore

STIX Core functionalities
BSD 3-Clause "New" or "Revised" License
3 stars 3 forks source link

Error generation end2end test data for calibration spectra #306

Open samaloney opened 1 year ago

samaloney commented 1 year ago

The script to generate the data used for end2end testing is showing an error but not failing so we should fix the error and maybe fail the test on any error?

2023-01-18T14:10:59Z ERROR stixcore.processing.LBtoL0 91: Error processing file /private/var/folders/r1/6cz6bl3156s18tp0wqggp6mc0000gn/T/pytest-of-shane/pytest-51/LB/21/6/41/solo_LB_stix-21-6-41_0678240000_V01.fits for 21, 6, 41
Traceback (most recent call last):
  File "/Users/shane/Projects/STIX/STIXCore/stixcore/processing/LBtoL0.py", line 86, in process_tm_type
    level0 = tmp.from_levelb(levelb, parent=file.name)
  File "/Users/shane/Projects/STIX/STIXCore/stixcore/products/level0/quicklookL0.py", line 623, in from_levelb
    counts_rebinned = rebin_proportional(np.hstack(cur_counts), cin, c_out)
  File "/Users/shane/Projects/STIX/STIXCore/stixcore/products/common.py", line 252, in rebin_proportional
    start = cum_sum[np.ceil(i_place[:-1]).astype(int)]
IndexError: index 26 is out of bounds for axis 0 with size 26
2023-01-18T14:10:59Z ERROR stixcore.processing.LBtoL0 91: Error processing file /private/var/folders/r1/6cz6bl3156s18tp0wqggp6mc0000gn/T/pytest-of-shane/pytest-51/LB/21/6/41/solo_LB_stix-21-6-41_0678240000_V01.fits for 21, 6, 41
Traceback (most recent call last):
  File "/Users/shane/Projects/STIX/STIXCore/stixcore/processing/LBtoL0.py", line 86, in process_tm_type
    level0 = tmp.from_levelb(levelb, parent=file.name)
  File "/Users/shane/Projects/STIX/STIXCore/stixcore/products/level0/quicklookL0.py", line 623, in from_levelb
    counts_rebinned = rebin_proportional(np.hstack(cur_counts), cin, c_out)
  File "/Users/shane/Projects/STIX/STIXCore/stixcore/products/common.py", line 252, in rebin_proportional
    start = cum_sum[np.ceil(i_place[:-1]).astype(int)]
IndexError: index 26 is out of bounds for axis 0 with size 26
2023-01-18T14:10:59Z ERROR stixcore.processing.LBtoL0 93: index 26 is out of bounds for axis 0 with size 26
2023-01-18T14:10:59Z ERROR stixcore.processing.LBtoL0 93: index 26 is out of bounds for axis 0 with size 26
2023-01-18T14:10:59Z INFO stixcore.idb.idb 522: IDB loaded from /Users/shane/Projects/STIX/STIXCore/stixcore/data/idb/v2.26.35/idb.sqlite
samaloney commented 1 year ago

we should look into / fix this before the data release

samaloney commented 1 year ago

So this seems to be something to do with how we create the test data as when the source files are processed normally this error doesn't occur it only after we generate our own xml file that this happens so look like something is being reordered or missed.