Create / update stix fits reader to work with L1A/L1 and L2 fits files

[samaloney]

One common routine shared between imaging and spectroscopy side that returns similar if not identical structures for L1A and L1 files and soon L2 files

[samaloney]

I'd suggest something like this cpd = stx_fits_read('<some file>') then we could return the data and headers for all extensions in a simple structure so cpd.data.counts or cpd.data.header and then the same for the primary extension and others e.g. control, idb_version, and energy?

If if you only wanted certain extension/s cpd = stx_fits_read('<some file>', ext=['data', 'control']) I think the primary head is always necessary for time conversion etc?

What do people think @paolomassa @grazwegian ?

[grazwegian]

From my side this sounds good!

I can get to work on an update for this before the end of the year.

Yes, for most purposes you will need the primary header for time conversion.

I would guess that it might make sense to have the facility for the L2 corrections we are currently doing for the L1/A here so they are in the same place for spectroscopy and imaging. Though sometimes we will want to let people have the uncorrected data so they can do their own.

[paolomassa]

Fine also for me, thanks!

[samaloney]

So I guess in order of importance

1. Compressed Pixel Data
2. Spectrogram
3. Raw Pixel Data
4. QL data

I'll have a look at this next week and see if I can return a common structure that could be use in other routines.

[grazwegian]

From the Afternoon work session of GSW colocation on 29.03.22

First step: make imaging compatible with ‘stx_read_pixel_data_fits_file.pro’ for reading the fits files;

Second step: fill in a 'pixel_data' structure shared by both imaging and OSPEX

[paolomassa]

I've starting looking into 'stx_read_pixel_data_fits_file.pro' and I do not understand lines 100-101: data.counts_err = sqrt(data.counts_err^2. + data.counts) data.triggers_err = sqrt(data.triggers_err^2. + data.triggers)

Why do you need to add the counts/triggers to the errors?

Also, how are compression errors handled by this code? Thanks!

[grazwegian]

The errors in the files are the compression errors so these lines are adding in the stat error in quadrature.

[paolomassa]

Thanks @grazwegian!

I've tried to plan how to develop the code for reading the data and creating a structure shared by both OSPEX and imaging. Below a tentative solution:

• read the L1 fits file with 'stx_read_pixel_data_fits_file';
• define a structure called 'stx_pixel_spectrogram' (or with another name) containing the spectrograms for each pixel. This could be the structure shared by OSPEX and imaging. Specifically, the fields of the structure could be:
  • counts: array of dimension (number of energy channels, number of pixels, number of detectors, number of time bins) containing the counts measured by each pixel
  • counts_err: array of the same dimension as counts containing the errors on the measured counts (compression + poisson)
  • triggers: array of dimension (half of the number of detectors, number of time bins) containing the trigger values
  • triggers_err: array of dimension (half of the number of detectors, number of time bins) containing the errors on the trigger values (compression + poisson )
  • time_axis: stx_time_axis structure
  • energy_axis: stx_energy_axis_structure
  • rcr: array of dimension (number of time bins) containing the rate control regime status for each time bin
  • pixel_masks: array of dimension (number of pixel, number of pixel, number of time bins) contain the mask of the pixels used at each time bin
  • detector_masks: array of dimension (number of detectors, number of time bins) containing the mask of the detectors used at each time bin

Essentially, this structure is very similar to the 'data' structure built at the end of 'stx_read_pixel_data_fits_file'. I wouldn't add the following fields in 'stx_pixel_spectrogram' (that are now in 'data') since it seems that they are not used in other parts of the ground software (or that they are not needed):

• time
• timedel -> if I understood correctly, it should be the same as time_axis.duration
• control_index
• num_pixel_sets
• num_energy_groups

One possibility is to build the 'stx_pixel_spectrogram' structure inside 'stx_read_pixel_data_fits_file' and replace the 'data' structure. The other possibility, is to define it outside 'stx_read_pixel_data_fits_file'. If you agree with the proposed tentative solution, please tell me what you prefer about this.

Since imaging and OSPEX handle some corrections (ELUT correction and the background subtraction) in a different way, I don't think that they can share more than the structure I described above. But again, please, tell me if you have a different opinion on this.

Finally, I would construct a 'stx_pixel_data' structure just for imaging purposes from 'stx_pixel_spectrogram' and I would start from 'stx_pixel_data' for defining a visibility structure.

[ennosigaeus]

Discussed this today and decided to close it as done.