IP Camera Support

[myjtag]

Feature Request

Please support IP cameras, Because there are a wast camera selection and prices and performances are way more optimised than USB webcams, Also the Bandwidth requirement for USB is eliminated and we can use as many Cameras as our CPU can handle, also the cable length is not that Important either, the best part is that we can have Higher resolution and or higher refresh rate camera for selection, and the prices are very low for some models.

[markmaker]

Those are supported. Use the OnvifIPCamera

Report back if that is not what you mean.

Because there are a wast camera selection and prices and performances are way more optimised than USB webcams ... the best part is that we can have Higher resolution and or higher refresh rate camera for selection ...

Really? Can you back this up? :smile:

As for band-width, don't forget USB3.

_Mark

[myjtag]

Thanks to hear that it has support for IP cameras. I have just run a simple test with my android phone, using the IP webcam application, I can clearly see the video feed in the browser using this address http://192.168.1.3:8080/ or use this for raw video http://192.168.1.3:8080/video But I can not get any video in open pnp, uising any of the urls. Also I think these days the cell phone cameras are great and cheap for this type of work, since xiaomi has phones for around 50$ that have 4K main cameras! also they support lower resolution with 120Hz frame rate.

[markmaker]

I guess you have to find out the specification of these feeds. I don't know much about IP cameras, but I guess one thing to specify is the codec. If I'm not mistaken, OpenPnP currently only support MJPEG. Maybe there are settings available in your app to switch to MJPEG, or maybe the OpenPnP codec support could be extended in the future.

Note, you don't need/want 4k resolution. This is counter-productive, because a) excessive resolution makes computer vision slow (some algorithms have computation times that grow to the fourth order of the linear resolution!), b) the extra resolution is not needed for the precision that is required for pick and place, c) the extra resolution often comes at the cost of more compression artifacts, and while they may be okay for limited human perception, they can be disruptive for precision geometry computer vision.

Note, we are using sub-pixel computer vision where it matters, so we can get a lot from little resolution, provided the image does not suffer in compression.

Whether using a phone with an app from an app store, mounted on a machine is a good idea, I leave to you. :stuck_out_tongue_winking_eye:

However, before you proceed, measure the lag from actual physical event to decoded image. This is much more important than resolution or fps. Such a lag may not matter for a classical webcam that streams a show or the weather. But it does matter for us.

_Mark

[myjtag]

thanks, I think you should extend the IP camera support for open PNP, by the way what camera do you suggest, since I want to build a new machine that can handle small components, like 0201 and has high speed,

[markmaker]

by the way what camera do you suggest, since I want to build a new machine that can handle small components, like 0201 and has high speed

We still recommend using the ELP 720p camera, as it combines very good quality with reasonable resolution. It is proven to work with two cameras on the same USB 2.0 root hub. There are more advanced cameras, sure, but nobody has really documented them as better overall. For instance, I tested a global shutter 120 fps camera, it surely was faster, but it had serious flaws, clipping into false colors, for instance.

https://github.com/openpnp/openpnp/wiki/Build-FAQ#what-should-i-build

If you don't care about using two USB 2.0 ports on your computer, you can select the ELP 1080p camera, and use it at 720p with 60 fps (instead of 30fps for the recommended model). I do that on my machine.

As to 0201 components, I haven't heard of any users that pulled it off. Those are really, really small. Even the 0402 I use are so delicate. You would be a pioneer :sunglasses:

There is nothing in OpenPnP itself that would prohibit such a thing, though. Assuming that all components you use are equally smaller, you would just have your cameras a bit closer, and the controller steps per millimeter a bit higher.

If you still also mix in a few very large parts, it is equally no problem, because we have Vision Compositing a.k.a. Multi-Shot bottom vision.

_Mark

[myjtag]

thanks for sharing.

[markmaker]

If you or somebody takes the time to specify missing IP camera protocols and/or codecs, for cameras that are becoming affordable, tested for lag and usable with OpenPnP, then I'll keep this open. Otherwise I'll close it in a few days.

[myjtag]

I have tested it with some App's supporting IP camera protocols like ONVIF support, why Android phones? because they are cheap, have good cameras and have a good enough processor that can be used to handle image processing if open pnp Add support for it, Here is the list and the app link in the list 1- IP Webcam app https://play.google.com/store/apps/details?id=com.pas.webcam&hl=en_CA&gl=US

Also I have tested these IP cameras, with no lock EYE guard EG-IR93040 wifi ptz camera

[jwilo]

This is actually something I'd like to get around adding support for in the future, but realistically won't get to it for 6-9 months. The majority (read: essentially all) machine vision cameras don't support Onvif like IP CCTV cameras do, but typically support the GigE Vision protocol (https://en.wikipedia.org/wiki/GigE_Vision).

I'm planning a future build that may very well use GigE vision cameras, and thus this'll be something I'll need to do, but can't promise anything at this time.

There is however a problem in that GigE Vision is not open - typically this isn't a problem for most applications as camera vendors supply SDKs that work with all the big languages, that in turn implement GigE Vision, so the realistic approach to this is to use those, I feel. For example FLIR machine vision cameras use the FLIR/Teledyne Spinnaker SDK.

[jwilo]

Ah, I've done some work with Spinnaker recently in C#, but have just realised there is no Java support. Quick search shows the same situation with Cognex, so that pretty much rules out 2 of the biggest GigE camera vendors.

[jwilo]

...further thoughts, perhaps it's possible to call the .NET SDK from the JRE via something like this, but that's getting messy quickly. I'm sure there are other ways (short of porting OpenPnP to .NET!) if I spent more than a few minutes thinking about it.

Perhaps a better solution is to ask whether a license would be granted FOC to OpenPnP, such that a GigE vision driver can just be natively developed in Java.

I suspect A3 would be open to this due to this quote:

Anyone who has purchased GigE Vision compliant product and wishes to do their own unique one-off integration or application development may use the standard without a license for intra-company or intra-academic departmental use only. System integration companies may use the standard without a license to provide integration service to their customers.

Worth nothing though, that the OP's statement on cost is not necessarily true. MV IP cameras are typically big £££, or at least - much more expensive than the small ELP modules used by OpenPnP users.

the prices are very low for some models.

In hindsight of typing this out, I think I've changed my mind for my next build! The majority of 'serious' MV cameras are also available in USB 3, with the screw locking connector variant, for example - these.

[vonnieda]

the best part is that we can have Higher resolution and or higher refresh rate camera for selection,

This is a negative for OpenPnP. There is no benefit to pushing more pixels than we need for vision ops. It just means moving more data and using more CPU.

I think you should ask yourself what the benefit is of these cameras. Over the years we have tested dozens of cameras and there are good reasons we've settled on the 720p ELP camera: It's cheap, highly available worldwide, has enough resolution, has enough FPS, supports compressed frames, supports full UVC control, no restrictions or licenses to use, can mount many inexpensive lenses, no weird SDKs, etc.

What are the good reasons for using a more powerful and expensive camera?

Jason

[jwilo]

Hi Jason,

I can't speak for the OP, but in my case the only reasons for wanting to move away from the ELP modules are those of physical robustness, that IMO doesn't justify the development effort of supporting GigE vision, especially when the same sensors are available in the same package, just with USB3 instead. I agree on the resolution front, the majority of all the high end FLIR and Cognex cameras are available from 720p upwards too, rather than jumping straight into the 40MP deep end.

[ghost]

What are the good reasons for using a more powerful and expensive camera?

The good reasons are:

• ROI (much faster image transfer: small images for small parts)
• Use of good C-Mount lenses and not M12 (distortions!)
• External trigger capabilites, exact exposure control
• Flyby-Vision possible

These reasons were the deciding factor for me to stop using OpenPnP and try something else.

Java is operating system independent and therefore very universal, but when it comes to fast machine vision, it is simply not particularly suitable because most camera SDKs do not support it.

In addition, OpenPnP is probably only designed for frugal hobbyists and not for production use, and that is a good thing, otherwise the professional assembly machine manufacturers would no longer make any money at all.

:wink:

[vonnieda]

What are the good reasons for using a more powerful and expensive camera?

The good reasons are:

* ROI (much faster image transfer: small images for small parts)
* Use of good C-Mount lenses and not M12 (distortions!)
* External trigger capabilites, exact exposure control
* Flyby-Vision possible

Great insight, thanks! Great point about ROI.

It's easy to add a new Camera interface to OpenPnP, and accessing native code from Java is easy enough. The main thing stopping us from having ever supported fancier cameras is primarily SDK licensing.

All the ones I've looked at, and it has been awhile, had awful NDAs, licenses, closed source binaries, no right to distribute, etc. So, while we could certainly write a driver to support a given SDK, we couldn't include it in OpenPnP.

If someone knows of an IP camera that supports these kinds of features, has an open and free spec or SDK, and is affordable, please let me know. I'd be happy to write a driver for such a camera.

Re: ROI: One thing I've wanted to add to the capture system for a long time is a single frame capture function. This would allow us to specify per frame metadata like ROI, lighting, trigger, etc. and allow drivers to block until the frame is captured. I never added this because it didn't seem to be supported in UVC and, as mentioned, most of the other SDKs are closed.

FWIW, the ELP camera is available in a nice cubed aluminum case with a C mount. I have a few and they are solid.

https://www.aliexpress.us/w/wholesale-elp-camera.html?spm=a2g0o.home.search.0

Jason

[jwilo]

Thanks for the tip Jason on those enclosed ELP modules, I'd not seen those before!

[ian-arkver]

A good while ago I added a GStreamerCamera, but it got removed again because a necessary pom.xml update broke something else.

Some of these GigE camera vendors supply a gstreamer plugin.

If there's interest I could look at bringing GStreamerCamera back up to date and doing a new PR. It was a very basic implementation however. Everything had to be set up on the gstreamer side, with no controls back from OpenPnP to the source elements or indeed any elements in the pipeline. It might get you pictures, but wouldn't get you any of the other features you mention above.

[j90141355]

What are the good reasons for using a more powerful and expensive camera?

The good reasons are:

* ROI (much faster image transfer: small images for small parts)
* Use of good C-Mount lenses and not M12 (distortions!)
* External trigger capabilites, exact exposure control
* Flyby-Vision possible

Great insight, thanks! Great point about ROI.

It's easy to add a new Camera interface to OpenPnP, and accessing native code from Java is easy enough. The main thing stopping us from having ever supported fancier cameras is primarily SDK licensing.

All the ones I've looked at, and it has been awhile, had awful NDAs, licenses, closed source binaries, no right to distribute, etc. So, while we could certainly write a driver to support a given SDK, we couldn't include it in OpenPnP.

If someone knows of an IP camera that supports these kinds of features, has an open and free spec or SDK, and is affordable, please let me know. I'd be happy to write a driver for such a camera.

Re: ROI: One thing I've wanted to add to the capture system for a long time is a single frame capture function. This would allow us to specify per frame metadata like ROI, lighting, trigger, etc. and allow drivers to block until the frame is captured. I never added this because it didn't seem to be supported in UVC and, as mentioned, most of the other SDKs are closed.

FWIW, the ELP camera is available in a nice cubed aluminum case with a C mount. I have a few and they are solid.

https://www.aliexpress.us/w/wholesale-elp-camera.html?spm=a2g0o.home.search.0

Jason

请问能添加支持这家的GIGE摄像头吗？比如型号为MV-CA013-xxxx和MV-CA050-XXXX。在中国这些摄像头在二手市场保有量非常巨大，并且廉价。目前在中国个人开发制造的贴片机中使用频率非常高。他的网址是 https://www.hikvision.com/en/ 。期待openpnp获得更好的发展