twVolc / PyCamPermanent

Permanent PiCam (SO2) installation project software
GNU General Public License v3.0
2 stars 2 forks source link

Buffer size - possibly not working as expected? #29

Closed twVolc closed 1 year ago

twVolc commented 1 year ago

The buffer size option in Processing>Settings almost always ends in a warning being raised in the GUI when "Run" is clicked. It might be that the adequate buffer size is incorrectly calculated as I'm sure the warning often pops up even when the buffer is larger than required. I always just ignore this warning and typically things still seem to run fine (I usually set the buffer size to 1500).

The buffer is basically an empty array of data (or possibly a dictionary with multiple arrays for different datasets - e.g. optical flow, optical depth) that is used to store old data until we reach the point at which calibration is possible i.e. until after the time set for DOAS calibration. The processing then loops back through all of these data to calibrate them and generate emission rates.

ubdbra001 commented 1 year ago

Loks like the timedelta function thinks the value being provided is days rather than mins: image

Will fix it to interpret it as minutes instead

ubdbra001 commented 1 year ago

Also, bit nit picky, but you don't need an if statement there. You can just return the value of the comparison, i.e.: return img_buff_timespan >= datetime.timedelta(minutes = self.doas_fov_recal_mins) and it'll always give you True or False

twVolc commented 1 year ago

It's good to have the nitpicking - that's useful to know! And yes it looks like you've nailed the issue there, thanks for that. I'm not 100% sure what will happen if the buffer size isn't set to be sufficient - i.e. will the earlier data be just be overwritten and not processed? Possibly worth investigating.

And that's also making me think that is it just easier to automate this whole thing, rather than letting the user define the buffer size, just work out how big it needs to be to hold all data prior to calibration. I'm not sure why I decided to make this a user-defined variable... This part isn't urgent though - as long as we know the check is working properly and what happens if the buffer size is set too small.

ubdbra001 commented 1 year ago

Okay, I'll add a fix, add a test, and poke around to see what happens if the buffer is too small. I'll open a new issue with the results of what I find and we can have a chat about how to approach it (if needed) there.