Buffer of frames and python wrapper

[kasasxav]

Hey !

I am trying to implement PyVCAM in my project and I have two issues I wanted to discuss:

• Whenever I'm using start_live() with the external trigger mode, the method poll_frame gets stuck when there are no frames. This is quite dangerous because it's freezing my threads. I was wondering, is there some variable I can use from the python wrapper to check if there are new unread frames? I'm seeing some variables in the cpp and SDK that make me think it's possible, but they are not available from the wrapper, as far as I know.

• I would also like to be able to pull all unread frames from the buffer into a tuple or array, is that possible? Again doing a for loop in my thread wouldn't be my favorite thing, I could do that but I would need some variable to know how many frames there are in the buffer that haven't been read yet.

Probably this is already possible somehow but I haven't managed to find a solution.

[stevebellinger]

Hi there.

1. For the poll_frame stuck problem. My first suggestion is to only call start_live when you know the external trigger source is active. If that's not possible, I can help you modify the wrapper to include a timeout on the poll_frame function so that the function never hangs.
2. I'm not entirely sure what you are looking for with the unread frames request. Can you elaborate on your use case? How many frames do you want to buffer and how often? In live mode, PVCAM buffers up to 16 frames, but PyVCAM only gives you access to the latest frame. This behavior can be modified. Alternatively, perhaps you can switch to sequence mode? Sequence mode can buffer more than 16 frames and can give you access to all buffered frames.

[kasasxav]

Thank you for the quick reply :)

1. I tried a bit in that direction already but the problem is that I don't always know when it stops, so then it's hanging in the end and I can't terminate it. The problem is that the camera is synchronized with other devices so it's not always trivial to know when it will stop acquiring images since it also depends on how many there are in the buffer. I wouldn't mind changing the wrapper if you give me some guidelines, that would be very nice!

2. So when I take a scan it can be around 800 frames, and I have a thread that saves them into memory continuously. Usually, with my other camera, I was calling a function that got the ones that I hadn't read yet by the time of execution since I might call the function to grab the frames slower than the camera grabs them. So I'm not sure how to proceed in this case. Do you think sequence mode is compatible in this case? I guess the issue is that sometimes I don't know how many frames there will be but I could run sequence mode in a cycle, I guess.

[stevebellinger]

No problem.

1. I can do the change, but not until tomorrow at the earliest. If you want to give it a shot, take a look at pvcmodule.cpp:767. You want to comment out the whole ALLOWTHREADS section and throw an exception (return NULL) if camInstance.newData is false.
2. Sequence mode is not going to be suitable for this many frames. It is possible to alter PyVCAM to allow access to the full live mode circular buffer and even increase the depth of the buffer. But before we go that route, do you have a notion for how the deep the buffer needs to be? Is it possible to tune your application so that poll_frame always gets called faster than the frame rate?

[kasasxav]

1. Thanks ! I will try a bit as well, thank you for the direction.
2. I think if we fix 1., probably by being able to read the 16 frames buffer it might be good enough, I should check. The thread goes quite fast, it's just that if the exposure is 10ms then it's easy to lose frames if I can't access the full circular buffer.

[stevebellinger]

Sounds good. If you don't mind, please post your work in a branch so that I can follow along and help.

[kasasxav]

Then now the only thing would be to have a function get_frames to return all frames in the buffer, and we already have get_latest_frame (poll_latest_frame) for getting the last one, which I use for display (which I took from the branch poll_latest_frame)

[kasasxav]

Let me kow about this second one, I haven't done pr yet. So far I think I managed to not reset the queue but still doesn't return all frames available.