search

joshdoe / gst-plugins-vision

GStreamer plugins related to the field of machine vision
Other
133 stars 50 forks source link

Pleorasrc stream is found but fails to start from JAI GO-5100-PGE camera module, eBUS SDK 6.1.8, GStreamer 1.14.5 #44

Closed bsset3 closed 3 years ago

bsset3 commented 3 years ago

Have successfully compiled/installed the gstpleora6 module on Linux, Ubuntu 18.04 (AARCH64, Nvidia JETSON Nano board). The camera is up and working (eBUS player have no problem catching image stream over GigE Vision (PureGEV) (camera has separate IP from board)

the module was compiled with GCC 7.5.0. I had to add the -DUNIX compiler flag. link.txt The camera settings used, which were also loaded in a separate trial (with the plugin config-file= param) are enclosed. This trial yielded basically the same result. test.pvcfg.gz

gst-launch-1.0 pleorasrc ! videoconvert ! autovideosink Setting pipeline to PAUSED ... 0:00:00.088070921 8470 0x558683b860 WARN xcontext xvcontext.c:534:gst_xvcontext_check_xshm_calls: MIT-SHM extension check failed at XShmAttach. Not using shared memory. 0:00:00.105536122 8470 0x558683b860 WARN xcontext xvcontext.c:534:gst_xvcontext_check_xshm_calls: MIT-SHM extension check failed at XShmAttach. Not using shared memory. 0:00:03.500491386 8470 0x558683b860 WARN pleorasrc gstpleorasrc.cpp:1520:gst_pleorasrc_get_pvbuffer: error: Failed to retrieve buffer in timeout (1000 ms): 0x001e, '' 0:00:03.566564863 8470 0x558683b860 WARN basesrc gstbasesrc.c:3452:gst_base_src_start: error: Failed to start 0:00:03.566688824 8470 0x558683b860 WARN basesrc gstbasesrc.c:3807:gst_base_src_activate_push: Failed to start in push mode 0:00:03.566722992 8470 0x558683b860 WARN GST_PADS gstpad.c:1149:gst_pad_set_active: Failed to activate pad ERROR: Pipeline doesn't want to pause. ERROR: from element /GstPipeline:pipeline0/GstPleoraSrc:pleorasrc0: Failed to retrieve buffer in timeout (1000 ms): 0x001e, ''

Any clue as to what may be the issue?

joshdoe commented 3 years ago

Try running with higher debug output, set environment variable "GST_DEBUG=pleorasrc:9", and share that output.

bsset3 commented 3 years ago

0:00:00.064017758 10724 0x55b6e5dec0 DEBUG pleorasrc gstpleorasrc.cpp:1399:gst_pleorasrc_get_caps: The caps after filtering are video/x-bayer, format=(string){ bggr16, grbg16, rggb16, gbrg16 }, endianness=(int)1234, bpp=(int){ 16, 14, 12, 10 }, width=(int)[ 1, 2147483647 ], height=(int)[ 1, 2147483647 ], framerate=(fraction)[ 0/1, 2147483647/1 ]; video/x-bayer, format=(string){ bggr, grbg, rggb, gbrg }, width=(int)[ 1, 2147483647 ], height=(int)[ 1, 2147483647 ], framerate=(fraction)[ 0/1, 2147483647/1 ]; video/x-raw, format=(string){ GRAY16_LE, GRAY16_BE, GRAY8, UYVY, YUY2, RGB }, width=(int)[ 1, 2147483647 ], height=(int)[ 1, 2147483647 ], framerate=(fraction)[ 0/1, 2147483647/1 ] Setting pipeline to PAUSED ... 0:00:00.131591954 10724 0x55b6e5dec0 DEBUG pleorasrc gstpleorasrc.cpp:1258:gst_pleorasrc_start: start 0:00:00.138551285 10724 0x55b6e5dec0 DEBUG pleorasrc gstpleorasrc.cpp:567:gst_pleorasrc_find_device: Finding device based on index: 0 0:00:01.143018645 10724 0x55b6e5dec0 DEBUG pleorasrc gstpleorasrc.cpp:670:gst_pleorasrc_find_device: Info for device that will be opened: 0:00:01.143524074 10724 0x55b6e5dec0 DEBUG pleorasrc gstpleorasrc.cpp:447:gst_pleorasrc_print_device_info: Found device 'GO-5100C-PGE 00:0c:df:0a:14:33 [10.100.14.34]' 0:00:01.143874916 10724 0x55b6e5dec0 DEBUG pleorasrc gstpleorasrc.cpp:475:gst_pleorasrc_print_device_info: Device found on network interface 'eth0', MAC: 00:04:4b:e5:ac:e7, IP: 10.100.14.33, Subnet: 255.255.255.0 0:00:01.144254143 10724 0x55b6e5dec0 DEBUG pleorasrc gstpleorasrc.cpp:489:gst_pleorasrc_print_device_info: GEV device: MAC: 00:0c:df:0a:14:33, IP: 10.100.14.34, S/N: U510352 0:00:01.146196429 10724 0x55b6e5dec0 DEBUG pleorasrc gstpleorasrc.cpp:831:gst_pleorasrc_open_device: Trying to connect to device as controller 0:00:01.367787482 10724 0x55b6e5dec0 DEBUG pleorasrc gstpleorasrc.cpp:840:gst_pleorasrc_open_device: Connected to device as controller 0:00:01.367869099 10724 0x55b6e5dec0 DEBUG pleorasrc gstpleorasrc.cpp:890:gst_pleorasrc_open_stream: Using connection ID '10.100.14.34' 0:00:01.381759738 10724 0x55b6e5dec0 DEBUG pleorasrc gstpleorasrc.cpp:899:gst_pleorasrc_open_stream: Opening GEV stream in unicast mode 0:00:01.384231046 10724 0x55b6e5dec0 DEBUG pleorasrc gstpleorasrc.cpp:931:gst_pleorasrc_setup_stream: Stream created for device 0:00:02.550608003 10724 0x55b6e5dec0 DEBUG pleorasrc gstpleorasrc.cpp:967:gst_pleorasrc_setup_stream: Packet size is currently 1476 0:00:02.556789763 10724 0x55b6e5dec0 DEBUG pleorasrc gstpleorasrc.cpp:1266:gst_pleorasrc_start: Starting pipeline 0:00:02.569914603 10724 0x55b6e5dec0 DEBUG pleorasrc gstpleorasrc.cpp:1282:gst_pleorasrc_start: Opened as controller, so send AcquisitionStart command 0:00:02.570111795 10724 0x55b6e5dec0 DEBUG pleorasrc gstpleorasrc.cpp:1296:gst_pleorasrc_start: Lock streaming-related parameters 0:00:03.585663220 10724 0x55b6e5dec0 WARN pleorasrc gstpleorasrc.cpp:1520:gst_pleorasrc_get_pvbuffer: error: Failed to retrieve buffer in timeout (1000 ms): 0x001e, '' ERROR: Pipeline doesn't want to pause. ERROR: from element /GstPipeline:pipeline0/GstPleoraSrc:pleorasrc0: Failed to retrieve buffer in timeout (1000 ms): 0x001e, '' Additional debug info:

I am afraid this does not give very much more information, but at least we see that the camera unit is discovered. By the way, increasing timeout from 1-10s has no impact, obviously the timeout error is a symptom of something else.

NHarishGit commented 3 years ago

@joshdoe I am facing the same issue now too. GStreamer v 1.18.1, eBUS v 6.1.4.5137, VS 2019 v 16.8.1, Win 10 x64 platform

joshdoe commented 3 years ago

Are you letting the camera free run or doing triggering? Most times I've seen that when I've configured external triggering but the trigger source isn't applied correctly, or when the packet size is misconfigured, but I don't think either is the case here.

I do notice your config file specifies BayerRG8, which videoconvert won't handle, you'll have to use bayer2rgb or my bayer2gray, but that shouldn't be causing this timeout issue.

joshdoe commented 3 years ago

@joshdoe I am facing the same issue now too. GStreamer v 1.18.1, eBUS v 6.1.4.5137, VS 2019 v 16.8.1, Win 10 x64 platform

Hmm, maybe you're suggesting the issue is with GStreamer 1.18? I haven't changed pleorasrc in over a year. I can test soon with 1.18.

NHarishGit commented 3 years ago

@joshdoe I am facing the same issue now too. GStreamer v 1.18.1, eBUS v 6.1.4.5137, VS 2019 v 16.8.1, Win 10 x64 platform

Hmm, maybe you're suggesting the issue is with GStreamer 1.18? I haven't changed pleorasrc in over a year. I can test soon with 1.18.

Unsure, if it is because of GST v 1.18, because my log looked like @bsset3's. timing out and I do NOT have camera triggering it is free flow.

bsset3 commented 3 years ago

The camera is set to continous run, no external trigger is used (Triggermode=Off). I have tried both fixed packet size (1476) or auto-negotiation.

NHarishGit commented 3 years ago

Have successfully compiled/installed the gstpleora6 module on Linux, Ubuntu 18.04 (AARCH64, Nvidia JETSON Nano board). The camera is up and working (eBUS player have no problem catching image stream over GigE Vision (PureGEV) (camera has separate IP from board)

the module was compiled with GCC 7.5.0. I had to add the -DUNIX compiler flag. link.txt The camera settings used, which were also loaded in a separate trial (with the plugin config-file= param) are enclosed. This trial yielded basically the same result. test.pvcfg.gz

gst-launch-1.0 pleorasrc ! videoconvert ! autovideosink Setting pipeline to PAUSED ... 0:00:00.088070921 8470 0x558683b860 WARN xcontext xvcontext.c:534:gst_xvcontext_check_xshm_calls: MIT-SHM extension check failed at XShmAttach. Not using shared memory. 0:00:00.105536122 8470 0x558683b860 WARN xcontext xvcontext.c:534:gst_xvcontext_check_xshm_calls: MIT-SHM extension check failed at XShmAttach. Not using shared memory. 0:00:03.500491386 8470 0x558683b860 WARN pleorasrc gstpleorasrc.cpp:1520:gst_pleorasrc_get_pvbuffer: error: Failed to retrieve buffer in timeout (1000 ms): 0x001e, '' 0:00:03.566564863 8470 0x558683b860 WARN basesrc gstbasesrc.c:3452:gst_base_src_start: error: Failed to start 0:00:03.566688824 8470 0x558683b860 WARN basesrc gstbasesrc.c:3807:gst_base_src_activate_push: Failed to start in push mode 0:00:03.566722992 8470 0x558683b860 WARN GST_PADS gstpad.c:1149:gst_pad_set_active:pleorasrc0:src Failed to activate pad ERROR: Pipeline doesn't want to pause. ERROR: from element /GstPipeline:pipeline0/GstPleoraSrc:pleorasrc0: Failed to retrieve buffer in timeout (1000 ms): 0x001e, ''

Any clue as to what may be the issue?

@bsset3 can you please double check eBUS v is 6.1.4. OR 6.1.8 ? Just curious to know why do I have an older version)

bsset3 commented 3 years ago

My version of eBUS SDK is 6.1.8 (build 5374) You have a similar problem with 6.1.4, I understand @NHarishGit.

bsset3 commented 3 years ago

It turns out (under Linux) I am having a problem with eBUS SDK libraries not porting all needed library symbols (using the LD_LIBRARY_PATH environment variable) I have to preload some libraries, by: LD_PRELOAD="libPvVirtualDevice.so libPvAppUtils.so"

The last part of the full DEBUG output is:

0:00:02.634243162 19810 0x55a2a464c0 WARN pleorasrc gstpleorasrc.cpp:1526:gst_pleorasrc_get_pvbuffer: Failed to get buffer: 0x0b03, '' 0:00:02.899788227 19810 0x55a2a464c0 LOG pleorasrc gstpleorasrc.cpp:1221:gst_pleorasrc_pixel_type_to_gst_caps_string: Matched pixel type 17301513 to caps video/x-bayer, format=(string)rggb 0:00:02.900063962 19810 0x55a2a464c0 DEBUG pleorasrc gstpleorasrc.cpp:1411:gst_pleorasrc_set_caps: The caps being set are video/x-bayer, format=(string)rggb, width=(int)2464, height=(int)2056, framerate=(fraction)30/1 Pipeline is live and does not need PREROLL ... 0:00:02.900725645 19810 0x55a2a410a0 DEBUG pleorasrc gstpleorasrc.cpp:1391:gst_pleorasrc_get_caps: The caps before filtering are video/x-bayer, format=(string)rggb, width=(int)2464, height=(int)2056, framerate=(fraction)30/1 0:00:02.900765073 19810 0x55a2a410a0 DEBUG pleorasrc gstpleorasrc.cpp:1399:gst_pleorasrc_get_caps: The caps after filtering are video/x-bayer, format=(string)rggb, width=(int)2464, height=(int)2056, framerate=(fraction)30/1 ERROR: from element /GstPipeline:pipeline0/GstPleoraSrc:pleorasrc0: Internal data stream error. Additional debug info: gstbasesrc.c(3055): gst_base_src_loop (): /GstPipeline:pipeline0/GstPleoraSrc:pleorasrc0: streaming stopped, reason not-negotiated (-4) ERROR: pipeline doesn't want to preroll.

In this case we have a new error code: x0b03 = TOO_MANY_CONSECUTIVE_RESENDS. The below link suggests some solutions. (It may be that the plugin doesn't account for all the new STREAM CONTROL options of the protocol. https://supportcenter.pleora.com/s/article/TOO-MANY-CONSECUTIVE-RESENDS

After following some of the recommendations I still get: pleorasrc gstpleorasrc.cpp:1221:gst_pleorasrc_pixel_type_to_gst_caps_string: Matched pixel type 17301513 to caps video/x-bayer, format=(string)rggb 0:00:09.524120840 24448 0x55acec68c0 DEBUG pleorasrc gstpleorasrc.cpp:1411:gst_pleorasrc_set_caps: The caps being set are video/x-bayer, format=(string)rggb, width=(int)2464, height=(int)2056, framerate=(fraction)30/1 Pipeline is live and does not need PREROLL ... 0:00:09.525229876 24448 0x55acec10a0 DEBUG pleorasrc gstpleorasrc.cpp:1391:gst_pleorasrc_get_caps: The caps before filtering are video/x-bayer, format=(string)rggb, width=(int)2464, height=(int)2056, framerate=(fraction)30/1 0:00:09.525318055 24448 0x55acec10a0 DEBUG pleorasrc gstpleorasrc.cpp:1399:gst_pleorasrc_get_caps: The caps after filtering are video/x-bayer, format=(string)rggb, width=(int)2464, height=(int)2056, framerate=(fraction)30/1 Setting pipeline to PLAYING ... ERROR: from element /GstPipeline:pipeline0/GstPleoraSrc:pleorasrc0: Internal data stream error. Additional debug info: gstbasesrc.c(3055): gst_base_src_loop (): /GstPipeline:pipeline0/GstPleoraSrc:pleorasrc0: streaming stopped, reason not-negotiated (-4) Execution ended after 0:00:00.000187244

bsset3 commented 3 years ago

Good news. By scanning through the pipeline setup, and fixing an omission in the code I was able to get the the stream up and running: The correct basic gst-launch pipeline of a pleorasrc must be: gst-launch-1.0 pleorasrc ! bayer2rgb ! autovideoconvert ! autovideosink

(You need autovideoconvert to convert from raw-video rgb to a format accessible to autovideosink)

This was not enough however, the g_assert on gstpleorasrc.cpp:1640 failed because the src->gst_stride is not correctly updated from src->pleora_stride after GST_LOG_OBJECT on line 1627.

To fix this I amended the code with an actual copy-over: (1627-1632): " GST_LOG_OBJECT (src, "Row stride not aligned, copying %d -> %d", src->pleora_stride, src->gst_stride);

src->gst_stride=src->pleora_stride;
*buf = gst_buffer_new_and_alloc (src->height * src->gst_stride);

"

After this the memory allocation works, and a stream can commence, though it is not always stable with default settings for larger resolution cams. You need to provide a config-file with a good settings file to pleorasrc to obtain stable streaming. I hope this tiny fix will be amended to your code (and to your basic execution explanation)

joshdoe commented 3 years ago

I'm reopening this, as I'd like to make sure this gets fixed.

I understand about the need for bayer2rgb (or bayer2gray), this applies not to just this set of plugins but any use of Bayer in GStreamer, perhaps there could be some example pipelines in documentation somewhere though.

I think I see the problem. src->gst_stride is set in gst_pleorasrc_set_caps, however, only if it's a standard video format handled by GstVideo, which Bayer isn't. I can replicate this error now. Your fix will cause issues in other cases, such as Mono8 when the width is not a multiple of 4, which that copying code is meant to handle. I'll make a fix shortly and push it.

bsset3 commented 3 years ago

I have made a slightly improved fix in my own fork of your repo.

tir. 24. nov. 2020 kl. 16:58 skrev joshdoe notifications@github.com:

Reopened #44 https://github.com/joshdoe/gst-plugins-vision/issues/44.

— You are receiving this because you modified the open/close state. Reply to this email directly, view it on GitHub https://github.com/joshdoe/gst-plugins-vision/issues/44#event-4033575083, or unsubscribe https://github.com/notifications/unsubscribe-auth/ARJCRJ3GXXAT4JCGCLBWISLSRPJZFANCNFSM4T3QTP7A .

joshdoe commented 3 years ago

Ah, I didn't see your fork. I made and pushed a fix to my pleora_bayer branch just now. One thing your commit doesn't handle is if the stride is not a multiple of 4, which bayer2rgb and GStreamer in general expects. I tested it with BayerGR8 and BayerGR12 with varying widths. If you'd like to make a fix to your branch and submit a merge request I'll accept it, otherwise I'll push my fix

bsset3 commented 3 years ago

I have tested your fix and it works also for me, I have incorporated it into a similar new branch on my own repo. (You may push your fix to your own master)

The travails for me is not yet completely over however; As I am incorporating your gstreamer plugin under the NVIDIA deepstream framework I can't really live with the LD_PRELOAD mechanism to avoid blacklisting of the plugin as mentioned above. ( https://github.com/joshdoe/gst-plugins-vision/issues/44#issuecomment-731253105) I have tried therefore to augment the cmake files to properly include all eBUS SDK library dependencies dynamically in the gstpleora6 module.

Like this: (cmake/modules/FindPleora.cmake:44-) find_library (Pleora_LIBRARY_BASE PvBase${_LIB_SUFFIX} ${Pleora_LIBRARY_DIR}) find_library (Pleora_LIBRARY_DEVICE PvDevice${_LIB_SUFFIX} ${Pleora_LIBRARY_DIR}) find_library (Pleora_LIBRARY_VIRTUAL_DEVICE PvVirtualDevice${_LIB_SUFFIX} ${Pleora_LIBRARY_DIR}) find_library (Pleora_LIBRARY_APPUTILS PvAppUtils${_LIB_SUFFIX} ${Pleora_LIBRARY_DIR}) set (Pleora_LIBRARIES ${Pleora_LIBRARY_BASE} ${Pleora_LIBRARY_DEVICE} ${Pleora_LIBRARY_VIRTUAL_DEVICE} ${Pleora_LIBRARY_APPUTILS})

And similarly adding to sys/pleora/CMakeLists.txt: set (LIBRARIES ....... ${Pleora_LIBRARIES} )

By this mechanism I have been able to add all required dynamic links in gstpleora6 EXCEPT for some reason the PvAppUtils library, from readelf -d: "..... 0x0000000000000001 (NEEDED) Shared library: [libPvBase.so.6.1.8.5374] 0x0000000000000001 (NEEDED) Shared library: [libPvDevice.so.6.1.8.5374] 0x0000000000000001 (NEEDED) Shared library: [libPvVirtualDevice.so.6.1.8.5374] ...... 0x000000000000001d (RUNPATH) Library runpath: [/opt/jai/ebus_sdk/linux-aarch64-arm/lib:] "

Why is CMAKE not including libPvAppUtils.so.x.x.x?

Addendum: I see that example applications in the eBUS SDK basically link successfully to almost all of the of the libPv*.so libraries. I attempted to replicate this by testing out an extensive link script for the module. Though several extra libs are included in the dynlink list, the library libPvAppUtils.so.x.x.x still refuses to be incorporated. gstpleora_linker_full.txt

ghazalehtrb commented 1 year ago

I have tested your fix and it works also for me, I have incorporated it into a similar new branch on my own repo. (You may push your fix to your own master)

The travails for me is not yet completely over however; As I am incorporating your gstreamer plugin under the NVIDIA deepstream framework I can't really live with the LD_PRELOAD mechanism to avoid blacklisting of the plugin as mentioned above. ( #44 (comment)) I have tried therefore to augment the cmake files to properly include all eBUS SDK library dependencies dynamically in the gstpleora6 module.

Like this: (cmake/modules/FindPleora.cmake:44-) find_library (Pleora_LIBRARY_BASE PvBase${_LIB_SUFFIX} ${Pleora_LIBRARY_DIR}) find_library (Pleora_LIBRARY_DEVICE PvDevice${_LIB_SUFFIX} ${Pleora_LIBRARY_DIR}) find_library (Pleora_LIBRARY_VIRTUAL_DEVICE PvVirtualDevice${_LIB_SUFFIX} ${Pleora_LIBRARY_DIR}) find_library (Pleora_LIBRARY_APPUTILS PvAppUtils${_LIB_SUFFIX} ${Pleora_LIBRARY_DIR}) set (Pleora_LIBRARIES PleoraLIBRARYBASE{Pleora_LIBRARY_DEVICE} PleoraLIBRARYVIRTUALDEVICE{Pleora_LIBRARY_APPUTILS})

And similarly adding to sys/pleora/CMakeLists.txt: set (LIBRARIES ....... ${Pleora_LIBRARIES} )

By this mechanism I have been able to add all required dynamic links in gstpleora6 EXCEPT for some reason the PvAppUtils library, from readelf -d: "..... 0x0000000000000001 (NEEDED) Shared library: [libPvBase.so.6.1.8.5374] 0x0000000000000001 (NEEDED) Shared library: [libPvDevice.so.6.1.8.5374] 0x0000000000000001 (NEEDED) Shared library: [libPvVirtualDevice.so.6.1.8.5374] ...... 0x000000000000001d (RUNPATH) Library runpath: [/opt/jai/ebus_sdk/linux-aarch64-arm/lib:] "

Why is CMAKE not including libPvAppUtils.so.x.x.x?

Addendum: I see that example applications in the eBUS SDK basically link successfully to almost all of the of the libPv*.so libraries. I attempted to replicate this by testing out an extensive link script for the module. Though several extra libs are included in the dynlink list, the library libPvAppUtils.so.x.x.x still refuses to be incorporated. gstpleora_linker_full.txt

Hello @bsset3, I noticed you were planning to use the plugin in deepstream, I was wondering if you got it to work and if you know a repo for that, thank you.