Other machine vision capture and processing plugins

[joshdoe]

I'm creating this issue to track similar GStreamer capture plugins that have been discovered. I don't have specific plans to try to merge them with this project at this time, but think that might be a good idea in some cases.

Capture

• gst-plugin-dalsa: Supports Teledyne DALSA GigE cameras
• gst-plugin-ueye: Supports IDS uEye cameras under Linux (we already do so under Linux, though this exposes more properties than our source)
• gst-plugin-flycap: Supports Point Grey (now FLIR) cameras under Linux, via the FlyCapture SDK
• gst-plugin-lumenera: Supports Lumenera cameras under Linux
• gst-plugin-spinnaker: Supports FLIR Spinnaker cameras under Windows and Linux (including Jetson)
• gst-pylonsrc: Supports Basler Pylon cameras under Linux (this has been merged with gst-plugins-vision)
• pylon_gstreamer: Supports Basler Pylon cameras under Linux, but not a direct GStreamer plugin, instead links Pylon via appsrc
• tiscamera: Supports The Imaging Source cameras under Linux
• gst-plugin-toupcam: Supports ToupTek cameras (primarily microscopy) Other machine vision manufacturers I've not found GStreamer support for include:
• Allied Vision Technologies (aka AVT, via Vimba SDK)
• Euresys: We support Picolo cards, but don't support Coaxlink API, but might work via GenTL support
• JAI: JAI SDK is used for older cameras, but support is ending and being replaced by the Pleora eBUS SDK

Processing

• gst-plugin-binning: Provides binning (downsampling and averaging)
• gst-opencv-debayer: Provides demosaicing of Bayer imagery using OpenCV, which is faster than bayer2rgb

And a shout out to @atdgroup, @MattsProjects, @TheImagingSource and @zingmars to keep them in the loop.

[MattsProjects]

Hi readers, I'm curious for your input :-). When I developed pylon_gstreamer, I actually specifically avoided creating a canned plugin on purpose. My reasoning was that a plugin is most useful when using gst-launch to run pipelines from the shell, but that is most useful only in the prototyping stage. I imagined that once an application moves into serious development, the user would use the GStreamer API to make their own customized program. And in that case, the appsrc API offers endless possibilities and control when interfaced with the camera vendor's API. Also, it is quite difficult to maintain a plugin over time because there are so many cameras and features to be supported. So porting everything to the user through the plugin interface can take a really long time (if you built a plugin, I'm sure you know :)). So my question is, are users really following this path of gst-launch-for-prototyping, api-for-development? Or are they taking gst-launch much farther into the final application? If a plugin is really being used a lot in the final application, then I may revisit my project... Thanks!

[zingmars]

Fwiw when I was creating our pylon plugin our app would use pipelines instead of appsrc. I don't know what the justification for that was (it was a C++ & QT port from an older Python & QT app, so it might be just that our devs didn't want to improve on that) though.

Looking at github, there do certainly appear to be a few cases of people doing the same (https://github.com/search?l=C%2B%2B&q=gst-launch-1.0&type=Code , https://github.com/search?l=C&p=6&q=gst-launch-1.0&type=Code) but they're fairly rare.

Having said all that, https://github.com/zingmars/gst-pylonsrc is a plugin, so I don't think there's really a necessity for your to modify your work and people can just select whatever suits their needs.

[joshdoe]

@MattsProjects and @zingmars , I use GStreamer plugins in several "final" applications, but use them in rapid prototyping via gst-launch almost as much. The only problem I've had to date is properly exposing camera attributes. GStreamer element properties must be known at first runtime, so dynamic properties of cameras can't be enumerated (plus there are so many). I've considered creating a camera-properties property which would be a GObject with camera properties. Or I could expose properties via GenICam's GenApi, but even then how do I do it in a GStreamer friendly way.

I now need to add a plugin for Basler Pylon, and am looking at gst-pylonsrc. I hate to create a fork, but I need to support Windows, which automake is terrible at supporting, and I might have a difference of opinion on how the camera properties should be exposed (if I figure out the right way to implement it).

I really should continue working on the GenTL plugin I started, however I got stuck at the piece dealing with the GenApi XML. The official GenICam library restricts redistribution I believe, so I'd have to roll my own I think.

[MattsProjects]

Hi joshdoe, Interesting use case to think about! I wonder, the GenAPI in pylon has a "CValuePtr" in addition to "CIntegerPtr, CEnumerationPtr, etc." With that, you can access the feature from strings, like ->ToString() and ->FromString("123"). I wonder if it's possible to expose some kind of generic feature through the plugin, where the properties are the feature name and a string to set the value... I'll try to experiment with this sometime...

[joshdoe]

I've merged pylonsrc into this project, thanks @zingmars!

@MattsProjects : generic properties is on my mind, just created an issue if you'd like to take a look.

[joshdoe]

Just added gst-plugin-spinnaker, gst-plugin-dalsa, gst-plugin-spinnaker, @thompd27

[thompd27]

@joshdoe thanks for the interest!

These repos are still a bit messy, but they do function (I use them on windows, linux, and jetson boards). If I have some time I'll commit a cleaned up version of the spinnaker plugin I'm using right now on my system. It has more support for changing camera settings from the command line. Generally the spinnaker plugin is more mature than the dalsa one (I never need to change settings on my dalsa cameras).

[joshdoe]

@thompd27 thanks for the info, I might be getting a FLIR camera to play with soon which caused me to find your repos. I'd happily take any of your code changes and test them out if/when I get the camera.

[joshdoe]

@thompd27, I should also mention that I have a Teledyne DALSA plugin in this repo as well using the Sapera SDK.