Aharoni-Lab / miniscope-io

Data formatting, reading, and writing from miniscopes
https://miniscope-io.readthedocs.io
GNU Affero General Public License v3.0
6 stars 2 forks source link

StreamDaq video output changes depending on queue size #40

Closed t-sasatani closed 3 months ago

t-sasatani commented 3 months ago

This only affects pytests at the moment, and a turnaround is written below (just freeze default runtime variables and use config properly). It seems the queue sizes of StreamDaq affect how the last few frames of a video are handled (noticed this when addressing @sneakers-the-rat 's review comment https://github.com/Aharoni-Lab/miniscope-io/pull/35#discussion_r1719448212). Making the queue size larger (which should typically be better for grabbing more frames) leads to losing a few frames, so I guess the termination process is doing something weird.

This should be fundamentally addressed by making sure to squeeze out all data to video soon. However I think we can freeze the runtime variables in tests at the moment to prioritize the tests to be functional.

Temporary turnaround

In config.py set the default buffer queue sizes to the following so pytest uses these numbers,

serial_buffer_queue_size: int = Field(
    10,
    description="Buffer length for serial data reception in streamDaq",
    json_schema_extra={"env": "MINISCOPE_IO_SERIAL_BUFFER"},
)
frame_buffer_queue_size: int = Field(
    5,
    description="Buffer length for storing frames in streamDaq",
    json_schema_extra={"env": "MINISCOPE_IO_FRAME_BUFFER"},
)
image_buffer_queue_size: int = Field(
    5,
    description="Buffer length for storing images in streamDaq",
    json_schema_extra={"env": "MINISCOPE_IO_IMAGE_BUFFER"},
)

and increase buffer size like the following in .env file if we need a larger queue.

MINISCOPE_IO_SERIAL_BUFFER=100
MINISCOPE_IO_FRAME_BUFFER=100
MINISCOPE_IO_IMAGE_BUFFER=100
t-sasatani commented 3 months ago

@sneakers-the-rat solved this in #35