Directly access camera frameBuffer_ attribute

[anntzer]

It would be useful if it was possible to directly access the frameBuffer_ attribute of Cam_InstanceT from the Python side (e.g. as a https://docs.python.org/3/c-api/memoryview.html, which requires no copying). My use case is to have on the one hand a GUI that displays the frames being acquired at a high rate (it's OK if some frames are missed by the display loop), while wanting to later save all the data to the hard drive (here, I assume that the entire movie stack can fit in the RAM while the acquisition is running). AFAICT, an easy way to achieve this would be to have the GUI loop just fetch and display the image last acquired whenever it is ready (possibly at a much lower rate than the actual frame rate), and at the end of the acquisition, copy the frameBuffer to the hard drive.

Or perhaps I missed an easier way to achieve that? Thanks in advance for your help.

[stevebellinger]

I'm very happy to assist. I suspect I am missing something about your use case. If the total number of frames can fit in RAM, a sequence collect as shown in the seq_mode.py test at line 16, can get you the frame data, albeit as a numpy object, which you can save to disk at your leisure. Can you help me understand why that is not sufficient?

[anntzer]

Thank you for your quick reply.

My understanding(?) is that poll_frame() will actually go through all recorded frames, regardless of how late the caller (the GUI display loop, in my case) is relative to the camera; but I'd rather have the display just drop whatever frames it was not quick enough to display, so that it keeps being more or less "real-time". So e.g. assuming that the camera acquires at (making up numbers) 200fps and my display loop can only display 50fps, I'd rather have the display loop just drop 3 intermediate frames between each actually displayed frame. (Of course, I could just repeatedly call pollframe() until reaching the last one acquired so far (assuming there's some way to check that?), but then I would need to also keep all these frames somewhere so that I can later save them to the hard drive, which is more work which seems unneeded as that data is in frameBuffer anyways.)

(Whether the frameBuffer is accessible as a memoryview or a numpy array does not matter much to me, but it is useful that it directly references the frameBuffer rather than being a copy, as a copy would divide by 2 the maximum buffer size than can be usefully held in memory).

[stevebellinger]

I think I understand now. We can make this happen in a branch. Here are my proposed changes.

1. Currently frame data is stored in a std::queue object which doesn't permit access to the latest data without popping older data off the queue. We should switch to std::deque which is more flexible in this regard. There will be updates throughout pvcmodule.cpp to use the new type, but it should be easy enough to work through.
2. Bind a new C++ method called poll_latest_frame which can get the latest frame without removing from the queue. This will get you your frames for display.
3. Remove the np.copy from the Camera object. This copy causes most of the PyVCAM debate and has a long history which I won't get into. But in your particular case it's not needed and you are absolutely right that it will allow for a longer sequence. Possibly the copy will become optional in master at a later date, but it is very easy to get into trouble without it so I'm hesitant.

When the sequence completes, use the normal poll_frame function to get frames from oldest to newest and store to disk. This step must be accomplished prior to the finish_seq call which will deallocate the memory.

Does this plan meet your needs?

[anntzer]

I'm not really sure how switching to deque matters here, but other than that, yes, adding poll_latest_frame() which does not remove the frame from the queue would be great, thanks. (Or alternatively, poll_next_frame(), which waits until a new frame comes in, which may make things even easier to handle on the side of the display loop while avoiding unnecessarily redisplaying the same frame if the display loop is faster than the acquisition loop. But that's really just an optimization, and not a strict requirement.)

[stevebellinger]

Please see the pollLatestFrame branch.

1. You are right that deque wasn't needed. I missed that queue provided a method to get the newest element.
2. I kept the function names as get_latest_frame (C++) and poll_latest_frame (Python). These functions use the same wait for next frame scheme as the normal get_frame and poll_frame.
3. I provided a test script called latest_frame.py. Everything seems to be working as expected, but please let me know if anything is off.

Please keep me posted as to your progress integrating this branch.

[anntzer]

Apologies for not getting back to you earlier. I have since then been recommended by our camera vendor to just use the Micro-Manager support for PVCAM, which it turns out was sufficient for our needs (for now), and for which I could reuse older code I wrote for other cameras; therefore, I didn't have a chance to fully test your branch, sorry for that.

[stevebellinger]

Not a problem at all. Please let me know if I can be of any assistance in the future.