loadImageCalibration() is Called Excessively

[Cybis320]

The loadImageCalibration function is called twice per block. This function attempts to load the mask, flat, and dark images twice every 10 seconds. Among other things, the function checks if the mask is all white with the condition if np.all(mask.img == 255):. This seems a little unnecessary - shouldn't the function only be called once per run instead?

[dvida]

This is done to check whether any of those files have been updated on the fly so the capture doesn't have to be stopped. An option would be to check if they have been modified and then load them again. However, is it really a problem to read a few files every 10 seconds?

[Cybis320]

I'm really not sure if it's actually a problem. Checking for all white pixels repeatedly on the same image doesn't strike me as particularly efficient. I'm also not sure if it doesn't generate unnecessary IO operations which we don't want on flash based storage. It could be adding to the storage bandwidth bottleneck. Pure speculation on my part, but worst case scenario, at 1080p we are talking about 3x2MB per 5 sec, that 1.2 MB/s. That's the transfer speed of some of our worst storage devices. What processes would update the files on the fly? Is there anything that would update mask on the fly?

[dvida]

Operators can update both files after an initial run and calibration, and there is also DownloadPlatepar and DownloadMask which can pull the files from the server. This is done when the operators cannot do it themselves. A simpler solution would be just to modify the code to check the modification time to reduce IO.

[Cybis320]

Those changes can only occur in between runs, not in the middle of a run, right?

[dvida]

Scenario: An operator turns on the camera and it starts collecting images. The operator makes a mask and a platepar based on the first collected images. The operator then installs the files and lets the system run. At the end of the night, the system will produce good measurements. If we remove live reading, this will not work and a full manual reprocessing step will be necessary.

[satmonkey]

I can confirm what Denis says. The correct mask is needed for the stars count and eventually subsequent meteor detection trigger. Creation od the mask (and platepar) is usualy done during the run, at no staturation and if stars are present.

[Cybis320]

Good to know. I've always stopped and restarted capture to do that - not realizing it was even an option.

[Cybis320]

So, in this scenario, would it be sufficient to have the code look for the file(s) until found and then continue the run with the first found file(s)?

[dvida]

Nope, as the files can be updated in the night. Tracking the modification date is the best option really.