search

cta-observatory / cta-lstchain

LST prototype testbench chain
https://cta-observatory.github.io/cta-lstchain/
BSD 3-Clause "New" or "Revised" License
22 stars 77 forks source link

Change dtype of drs4 baseline values to int16 to make it work with pre-corrected EVBv6 data. #1182

Closed maxnoe closed 7 months ago

maxnoe commented 7 months ago

I repeat here what was discussed via email:

In EVBv5, similarly to what is done now, data was shifted up by 400 to allow for negative values after the pre-calibration by EVB. But in the EVBv5 data, this offset wasn't specified, so it wasn't subtracted automatically. Instead, we computed the relative DRS4 offset coefficients on this shifted data, which resulted in drs4 coefficients around 400, subtracting this offset and also resulting in drs4 coefficients being positive (we store them as uint16 then).

With the new data format, the offset and scale are included and always applied by the source immediately when reading. This results in values around 0, so the drs4 coefficient script also computes drs4 offsets around 0.

This would all be fine, if we wouldn't store them as uint16, clipping the values below 0.

So my proposal would be to change to int16 for the drs4 offsets. Since we have a 12 bit ADC, that's not an issue, we still can store all possible values, also for not-yet applied pre-calibration by EVB. The binary representation also won't change, since all values up to now have been positive and < 2**15.

So in short, I think we should adapt the computation script to store values as int16 and issue the warning only in case of non-precalibrated drs4 values.

maxnoe commented 7 months ago

Tested on the evbv6 night, gives this distribution of drs4 offsets, as expected close around 0:

evb6_baseline

maxnoe commented 7 months ago

Please don't merge yet, I didn't yet test if applying these corrections like this actually works.

moralejo commented 7 months ago

@maxnoe Any news on this? We will need it for EvB v6 data even if the difference won't be large...

maxnoe commented 7 months ago

I tested with a standard drs4 evbv5 run, it yields the expected values around 400.

I didn't yet check in any meaningful way the application of the calibration to evbv6 data, but I don't expect any issues there, since the data with the clipped drs4 corrections was already ok.

moralejo commented 7 months ago

I tested with a standard drs4 evbv5 run, it yields the expected values around 400.

I didn't yet check in any meaningful way the application of the calibration to evbv6 data, but I don't expect any issues there, since the data with the clipped drs4 corrections was already ok.

Ok, thanks!