Calibration kernel

[de-narm]

@sredford Today we've finally begun to work again. We've had a long discussion about the calibration and decided on several questions:

1. We already know that we need at least 1000 dark frames. Do they need to be successive?
2. In order to work in parallel and be efficient, we could only update the pedestals after a set amount of frames. Is this sufficient or should we recalculate frames which used outdated pedestals when they change?

[sredford]

Hello again,

1) Under the assumption that the pedestal is stable, 1000 dark frames should be plenty to calculate it and the frames do not have to be successive. Each frame is an independent measurement of the same underlying value. Try looking at the pedestal, error on the pedestal and pedestal RMS as s function of number of dark frames considered.

2) Once the pedestal is drifting it is more important to have 'recent' frames going into the calculation. There are a few ways to do this, as you have figured out:

• update the pedestal with every frame
• update the pedestal every now and again with 1000 new frames
• update the pedestal continually, but taking only 1 in every N frames. Whether 'every now and again' is sufficient depends on the drift speed, which depends on the conditions and detector parameters, so there is not a good rule. Is updating every frame too inefficient to work? Maybe you could implement a parametrisable 1 in every N frame solution?

Note on the dynamic pedestal correction: you still need the dark frames from the dedicated pedestal file to give you a first calculation of the pedestal. That's because (from what I've seen so far) the spread in pedestal values is large compared to the drift. When using a non-dark file to calculate the pedestal it's important to ignore signal (when a photon hits the pixel). To work out whether there was a photon, you need to do the already existing pedestal correction and then look if the ADC is < 100 away from 0. This is my current rough cut to get rid of signal. If the pedestal corrected ADC in this pixel in this frame is within 100 ADC of the current pedestal of this pixel, then add the raw ADC to the pedestal calculation. Otherwise ignore it.

Hope that helps, come back to me with your next questions, Cheers, Sophie

[de-narm]

@sredford We're currently trying to get an update with every frame working. While doing so, I've encountered another question concerning the pedestal map: The .pdf-file we've received speaks of 3 different pedestal maps, one for each gain stage, when calculating the photon count. Unitl now, we've just used one for all of them since it's never mentioned how they'd be different - are they?

[sredford]

Yes, the pedestal of each gain stage will be different. The reason is that the pedestal is the 'noise' of the readout, and the readout chain of each gain stage is different (a different series of capacitors etc).

We gave you an 'allpede' pedestal file, which should contain 2999 frames. The first 1000 are dark frames for G0, the next 1000 are dark frames for G1 and the final 999 are dark frames for G2.

Calculate the three pedestals per pixel from these frames. The pedestal of G0 should be a few thousand, the pedestals of G1 and G2 should be much higher, around 15000. The difference is because in G0 we have a CDS stage, so 'zero means zero'. In G1 and G2 the CDS isn't used, so 'zero' is actually the maximum 14 bits and the more signal, the lower the ADC.

I think you can only update the G0 pedestal with frames from a signal dataset. The reason is that there is no measured noise peak in G1 and G2, since the pixel only switches to these gains in the case that there is signal.