Some simulations require two readouts

[hugobuddel]

%matplotlib inline
from matplotlib import pyplot as plt
import scopesim as sim

def do_simulation(extra_readout):
    star = sim.source.source_templates.star()
    cmd_l = sim.UserCommands(use_instrument="METIS", set_modes=["img_lm"])
    metis_l = sim.OpticalTrain(cmd_l)
    metis_l.observe(star, update=True)
    if extra_readout:
        _ = metis_l.readout(exptime=100.)[0]
    result_l = metis_l.readout(exptime=100.)[0]
    return result_l

result_noextra = do_simulation(extra_readout=False)

result_extra = do_simulation(extra_readout=True)

plt.figure(figsize=(10, 5))
plt.subplot(121)
plt.imshow(result_extra[1].data, origin='lower', extent=(-1.6, 1.6, -1.6, 1.6))
plt.xlabel("[arcsec]")
plt.ylabel("[arcsec]")
plt.title(r"With extra readout")
plt.subplot(122)
plt.imshow(result_noextra[1].data, origin='lower', extent=(-1.6, 1.6, -1.6, 1.6))
plt.xlabel("[arcsec]")
plt.ylabel("[arcsec]")
plt.title(r"Without extra readout")
plt.savefig("extrareadoutornot.png")

gives

So in this case it is necessary to call readout twice.

Both images should look like the left image.

[hugobuddel]

The first commit where this double-readout is needed is 92d8bc9c0de871f61d6d677198982f43c5fbaad5 from April 11 which has as message

Issue warning for mindit but don't actually change anything

On that commit, AutoExposure says

astar.scopesim.effects.electronic - Exposure parameters: DIT=0.000 s NDIT=11170144 astar.scopesim.effects.electronic - Total exposure time: 100.000 s

while on the previous commit it says

astar.scopesim.effects.electronic - Exposure parameters: DIT=0.040 s NDIT=2499 astar.scopesim.effects.electronic - Total exposure time: 99.960 s

main does not output any message with the default log settings

So it is clear that something goes wrong with the determination of DIT/NDIT.

Note that there is also an off-by-one error in NDIT in the last 'good' commit.

[teutoburg]

Maybe this is related to why some of the test I added in #428 currently fail?

[hugobuddel]

It works the second time because Detector.reset() does:

    def reset(self):
        """Reset internal HDU data to all-zeros-array."""
        self._hdu.data = np.zeros_like(self._hdu.data)

After the first time, self._hdu.data is changed from float to int16 at the end of the optical train by Quantization. Then the second time around, the zeros_like keeps it as int16 at the beginning of the optical train.

So it seems e7f9a3927ddc6ac539ca41f80c6ee32490226d98 is incorrect; the detector should always reset to float 0's.

However, that only explains why it works the second time. That does not yet explain why it does not work the first time around.

It seems that it fails the first time because the AutoExposure thinks that DIT and NDIT are given (and set to 1), which they are not, so AutoExposure does nothing. Quantization however thinks NDIT is 1 so it will quantize, leading everything to saturate (since NDIT should actually be about 2500, see above), and the image shows nothing. (And then everything seems fine the second time around because the detector is quantized already.)

So yeah, probably related to why tests in #428 fail.

[teutoburg]

I think the largest issue here is that the DIT & NDIT are not properly propagated between AutoExposure, SummedExposure and Quantization. I think once we solve the whole cmds & meta crap, issues like this might (almost) solve themselves. But maybe we can find a patch that works in the meantime...

[hugobuddel]

Yeah, I'll create something hackish that should work for now.