add support for pco cameras

[carandraug]

Requested at SPAOM 2019. They have a linux sdk which is free software under GPL v2+. Vendor offered to lend us two cameras to help us add support for them. There is also a separate windows sdk but it's not clear whether its interface is the same as the linux one.

[iandobbie]

The McGill team is trying to add support for a PCO camera, see cockpit issue https://github.com/microscope-cockpit/cockpit/issues/877.

[iandobbie]

@joris-afk I suggest you bring you questions about implementing an interface to the PCO camera here.

[joris-afk]

Hi, just a small question for the implementation of the Camera In the guide: https://python-microscope.org/doc/get-involved/new-device, it says "When a device is not controlled via serial it is most likely controlled via some vendor provided SDK". How is it different than if I call the PCO library from python? In the case of this implementation, I found many of the abstract functions to implement already implemented according to this guide: https://www.excelitas.com/file-download/download/public/102695?filename=pco_python_Manual.pdf . Therefore my current version of the code is just calling the respective function.

[joris-afk]

I was also wondering for getter outputs, do I still need to typecast them to the their respective Microscope class even if they already return as tuples from the PCO library's functions?

[iandobbie]

Sounds like you are getting things to work. The comment about serial device is because almost anything other than cameras presents as a serial device. Things like filter wheels, stages, light sources etc...

Cameras are generally the most complex devices to interface as they require complex configuration, such as trigger modes and types, exposure times, gains, bits depths etc... They also require high performance readout of data and transfer off to the connecting code. If the vendor provided python code is useful and maintained then plumb into that (see the ximea driver for an example). Unfortunately we have also found many instances where such code is far from a first class citizen and the only way to work out what you ned to do is to read the c documentation and follow the python calls down into the actual vendor provided dynamic library to work out what you should pass and what it might return.

As to you other question, yes you do need to cast output to the same types etc as specified in the ABC as this is what allows any code (such as cockpit) to utilize the device as a camera without adding device specific code.

[carandraug]

On Tue, 17 Oct 2023 at 17:12, Joris Ah-Kane wrote:

Hi, just a small question for the implementation of the Camera In the guide: https://python-microscope.org/doc/get-involved/new-device, it says "When a device is not controlled via serial it is most likely controlled via some vendor provided SDK". How is it different than if I call the PCO library from python? In the case of this implementation, I found many of the abstract functions to implement already implemented according to this guide: https://www.excelitas.com/file-download/download/public/102695?filename=pco_python_Manual.pdf . Therefore my current version of the code is just calling the respective function.

This is addressed a bit below:

Note

Some vendors provide Python bindings to their SDKs which may or may not be worth using. Often they are undocumented thin wrappers to their C library and if you use them, not only will you have to deal with undocumented behaviour from their C library you will also have to deal with the undocumented idiosyncrasies of their wrapper.

So that's for you to judge. I'm yet to not regret using a vendor Python library.

[iandobbie]

Having had a quick glance through the documentation the python library is pretty sparse and included this gem....

The object sdk allows direct access to all underlying functions of the pco.sdk.
>>> cam.sdk.get_temperature()
{'sensor temperature': 7.0, 'camera temperature': 38.2, 'power ←-
temperature': 36.7}
All return values from sdk functions are dictionaries. Not all camera settings are covered by the
Camera class. Special settings have to be set directly by calling the respective sdk function.

So basically the python library allows you to open the camera, set exposure time and a few basic readout parameters but all the complexity is hidden in the sdk class which is (surprise!) not documented!

[joris-afk]

Alright, so now knowing that I'll most likely follow the python calls in the vendors code, the structure of of the pco.Camera class, including its own file need 7 python files (1 main and 6 helpers) to for full access to its capabilities. One of those helpers is the general dll file reader such that the other classes can have access to one or more .dll files or .so files in the pco library or to free them. I see that only three of each type (dll and so) are relevant to the reader.

From what I understood from the "Support for New Device", I would then need to make 6 Python files to put under _microscope.wrappers just for the dlls and so, and then what about the 7 python files for the camera? Do I just combine them all in one big Python file? Or maybe even use the main camera code as base for the abstract Microscope functions to implement and have a seperate file combining the 6 other helpers.

What do you suggest?

[joris-afk]

Actually, nevermind, the only relevant code that would need to be traced is technically just pco.Camera(). Since the other ones are mostly just helper classes, a valid approach can also be to import all of them from pco into the code we'll write under the microscope.stages and that could be a workaround to dealing with the c libraries and sdk step.

What are your thoughts? Because I honestly don't believe we will finish the camera by the end of our project time otherwise.

[iandobbie]

I hope we didn't put you off working on this. I would start with the basic require functionality. For a generic camera I think the vital functions are:

1) Able to grab and image from the camera and put in in the queue to be push out to cockpit or whatever code you are using.

2) set and get exposure time and readout time or total cycle time so you know how frequently you can trigger it.

3) Set a trigger. It is helpful to have both software and hardware triggers but often easier to start with software, but cockpit basically can't do experiments without a hardware trigger.

You will have to fake some functions required by the abstract base class, such as setting roi etc... You can easily set these to just pass.

Then you can test things and get them working. Once this basis is working you can start adding functionality like selecting ROI's binning readout modes etc... depending on what functionality the camera possesses.

[carandraug]

Speaking with Mike Strauss, Joris and Aasmaa code is on the master branch of Aasmaa fork at https://github.com/aasmaa11/microscope/ (this has rudimentary PCO camera support and some improvements to ASI stage)

[iandobbie]

Seems that the two branches diverged on Oct 24th 2023.

[iandobbie]

I will create a new branch with the pco commits and link to it here.

I will reach out tot he pco reseller here and try to borrow a camera to test some of this code on. If they have snaps etc working I think getting live to work will be relatively easy, although triggering is always and significant issue with live streaming .

[carandraug]

I've just had a quick glance at the pco code and I see that it uses the pco Python library, which is exactly what we really recommended to not do. Having been burned before by taking this approach, I'm really not keen on pulling this.

[iandobbie]

Without a closer look at the PCO python interface it is hard to tell, however the implementation defines a PandaCamera class, but doesn't seem to be specifically limited to this.

I think the lack of any implementation of trigger mode might be the reason it can't do live video.