search

BrettRD / ros-gst-bridge

a bidirectional ros to gstreamer bridge and utilities for dynamic pipelines
Other
130 stars 31 forks source link

Crash with no error when executing simple and webrtc bins #45

Open gc-robotics opened 2 years ago

gc-robotics commented 2 years ago

Simple bin:
v4l2src ! videorate ! video/x-raw,width=640,height=360,framerate=1/15 ! rosimagesink ros-topic=/my_image

Webrtc bin: video_src_bin_descr: ' rosimagesrc ros-topic=/my_image ! videoconvert ! queue ! vp8enc deadline=1 ! rtpvp8pay ! application/x-rtp,media=video,encoding-name=VP8,payload=97 ! queue '

When running both, the execution stops with a "Process died" and not more info. Pls, see below:

[pipeline_node-1] [WARN] [1639817590.806759056] [gst_pipeline_node]: simple pipeline camera_input param_cb: unrecognised parameter "video_webrtc.autodial" [pipeline_node-1] [INFO] [1639817590.820707848] [gst_pipeline_node]: using node_id "1124" [pipeline_node-1] [INFO] [1639817590.825768052] [gst_pipeline_node]: using peer_id "2504" [pipeline_node-1] [INFO] [1639817590.831065913] [gst_pipeline_node]: using server "wss://webrtc.nirbheek.in:8443" [pipeline_node-1] [WARN] [1639817590.838665907] [gst_pipeline_node]: simple pipeline camera_input param_cb: unrecognised parameter "video_webrtc.stun_server" [pipeline_node-1] [WARN] [1639817590.846345276] [gst_pipeline_node]: simple pipeline camera_input param_cb: unrecognised parameter "video_webrtc.element_name" [pipeline_node-1] [WARN] [1639817590.854062302] [gst_pipeline_node]: simple pipeline camera_input param_cb: unrecognised parameter "video_webrtc.audio_src_bin_descr" [pipeline_node-1] [WARN] [1639817590.861810786] [gst_pipeline_node]: simple pipeline camera_input param_cb: unrecognised parameter "video_webrtc.video_src_bin_descr" [pipeline_node-1] [WARN] [1639817590.869499894] [gst_pipeline_node]: simple pipeline camera_input param_cb: unrecognised parameter "video_webrtc.video_sink_bin_descr" [pipeline_node-1] [WARN] [1639817590.877246920] [gst_pipeline_node]: simple pipeline camera_input param_cb: unrecognised parameter "video_webrtc.audio_sink_bin_descr" [pipeline_node-1] [DEBUG] [1639817590.884992435] [gst_pipeline_node]: building initial webrtc pipes [pipeline_node-1] [DEBUG] [1639817590.890325973] [gst_pipeline_node]: webrtcbin name=sendrecv bundle-policy=max-bundle stun-server=stun://stun.l.google.com:19302 rosimagesrc ros-topic=/my_image ! videoconvert ! queue ! vp8enc deadline=1 ! rtpvp8pay ! application/x-rtp,media=video,encoding-name=VP8,payload=97 ! queue ! sendrecv. [pipeline_node-1] [DEBUG] [1639817590.964455917] [gst_pipeline_node]: initial webrtc pipes built [pipeline_node-1] [DEBUG] [1639817590.970876329] [gst_pipeline_node]: transport connected callbacks [pipeline_node-1] [INFO] [1639817590.976477106] [gst_pipeline_node]: adding section video_webrtc [pipeline_node-1] [INFO] [1639817590.981895644] [gst_pipeline_node]: setting pipeline to PLAYING [pipeline_node-1] [DEBUG] [1639817590.983060747] [rcl]: Initializing ROS client library, for context at address: 0x24d61f50 [pipeline_node-1] [DEBUG] [1639817590.983339393] [rcl]: Using security: false [pipeline_node-1] [WARN] [1639817591.008244739] [rclcpp]: logging was initialized more than once [pipeline_node-1] [INFO] [1639817591.027066912] [gst_image_src_node]: stream_start at 1639817591027058787 [pipeline_node-1] [INFO] [1639817591.027635714] [gst_image_sink_node]: preparing video with caps 'video/x-raw, width=(int)640, height=(int)360, framerate=(fraction)1/15, format=(string)BGR, colorimetry=(string)1:1:7:4, interlace-mode=(string)progressive' [pipeline_node-1] [INFO] [1639817591.027976391] [gst_image_sink_node]: setcaps format string is BGR [pipeline_node-1] [INFO] [1639817591.028097172] [gst_image_sink_node]: setcaps n_components is 3 [pipeline_node-1] [INFO] [1639817591.028167432] [gst_image_sink_node]: setcaps bits is 8 [pipeline_node-1] [INFO] [1639817591.028227276] [gst_image_sink_node]: setcaps pixel_stride is 3 [pipeline_node-1] [INFO] [1639817591.030661961] [gst_image_src_node]: getcaps with filter 'NULL' [pipeline_node-1] [INFO] [1639817591.031012638] [gst_image_src_node]: waiting for first message [pipeline_node-1] [ERROR] [1639817591.544975275] [gst_pipeline_node]: remote says "ERROR peer '2504' not found" [pipeline_node-1] [INFO] [1639817592.668150164] [gst_image_sink_node]: stream_start at 1639817592668125216 [pipeline_node-1] [INFO] [1639817593.058464559] [gst_image_src_node]: preparing video with caps 'video/x-raw, format=(string)BGR, height=(int)360, width=(int)640, framerate=(fraction)0/1' [ERROR] [pipeline_node-1]: process has died [pid 51500, exit code -11, cmd '/home/ubuntu/robot/install/gst_pipeline/lib/gst_pipeline/pipeline_node --ros-args --log-level debug --ros-args -r __node:=gst_pipeline_node --params-file /home/ubuntu/robot/install/bootstrap/share/bootstrap/config/rasptank.config.yaml'].

Any idea what could it be going wrong? I can see that there is an ERROR logged on attempt to contact remote peer but I don't think that's what causing the crash.

Thanks.

gc-robotics commented 2 years ago

Maybe some more useful info in here?

0:00:46.924808372 60168 0xd10c360 DEBUG rosimagesrc rosimagesrc.cpp:488:rosimagesrc_create: create 0:00:46.924921080 60168 0xd10c360 DEBUG videopool gstvideopool.c:248:video_buffer_pool_alloc: alloc 691200 0:00:46.925099986 60168 0xd10c360 DEBUG GST_MEMORY gstmemory.c:138:gst_memory_init: new memory 0xffff6c024860, maxsize:691215 offset:0 size:691200 0:00:46.925823058 60168 0xcc88af0 DEBUG GST_BUS gstbus.c:542:gst_bus_timed_pop_filtered: got message 0xd280510, state-changed from queue0, type mask is 4294967295 0:00:46.926082016 60168 0xcc88af0 DEBUG GST_BUS gstbus.c:836:gst_bus_source_dispatch: source 0xd135440 calling dispatch with state-changed message: 0xd280510, time 99:99:99.999999999, seq-num 48, element 'queue0', GstMessageStateChanged, old-state=(GstState)GST_STATE_NULL, new-state=(GstState)GST_STATE_READY, pending-state=(GstState)GST_STATE_VOID_PENDING; 0:00:46.927254932 60168 0xcc88af0 DEBUG GST_BUS gstbus.c:842:gst_bus_source_dispatch: source 0xd135440 handler returns 1 Segmentation fault (core dumped)

gc-robotics commented 2 years ago

Some additional info, I got this out of some debugging:

Thread 18 "rosimagesrc0:sr" received signal SIGSEGV, Segmentation fault. [Switching to Thread 0xffffc0ff91e0 (LWP 72368)] __memcpy_generic () at ../sysdeps/aarch64/multiarch/../memcpy.S:193 193 ../sysdeps/aarch64/multiarch/../memcpy.S: No such file or directory.

BrettRD commented 2 years ago

The webrtc bindings are not currently stable.

Can you please make a minimal example of this? Your video source does not specify format, and your webrtc pipeline does not indicate your signalling protocol

Please also capture the ROS message (metadata only) immediately before the crash.

gc-robotics commented 2 years ago

This is the full webRTC bin:

webrtcbin name=sendrecv bundle-policy=max-bundle stun-server=stun://stun.l.google.com:19302 rosimagesrc ros-topic=/my_image ! video/x-raw,width=640,height=360,framerate=1/15,format=BGR,colorimetry=1:1:7:4,interlace-mode=progressive ! videoconvert ! queue ! vp8enc deadline=1 ! rtpvp8pay ! application/x-rtp,media=video,encoding-name=VP8,payload=97 ! queue ! sendrecv.

This is the input stream format: [pipeline_node-1] [INFO] [1640073812.045749889] [gst_image_sink_node]: preparing video with caps 'video/x-raw, width=(int)640, height=(int)360, framerate=(fraction)1/15, format=(string)BGR, colorimetry=(string)1:1:7:4, interlace-mode=(string)progressive'

and this is the rosimagesrc in the webrtcbin: [pipeline_node-1] [INFO] [1640073814.077561679] [gst_image_src_node]: preparing video with caps 'video/x-raw, format=(string)BGR, height=(int)360, width=(int)640, framerate=(fraction)1/15, colorimetry=(string)1:1:7:4, interlace-mode=(string)progressive'

and these are the last lines from the screen log:

[pipeline_node-1] [WARN] [1640073811.725688941] [gst_pipeline_node]: ignoring prop extra-controls type is <GType GstStructure (610406864)>
[pipeline_node-1] [WARN] [1640073811.731260280] [gst_pipeline_node]: ignoring prop pixel-aspect-ratio type is <GType gchararray (64)>
[pipeline_node-1] [DEBUG] [1640073811.736199277] [gst_pipeline_node]: adding param camera_input.v4l2src0.force-aspect-ratio Type.BOOL
[pipeline_node-1] [DEBUG] [1640073811.743582590] [gst_pipeline_node]: building param table bin
[pipeline_node-1] [INFO] [1640073811.748411431] [gst_pipeline_node]: adding section camera_input
[pipeline_node-1] [WARN] [1640073811.755292558] [gst_pipeline_node]: simple pipeline camera_input param_cb: unrecognised parameter "webrtc_bins"
[pipeline_node-1] [WARN] [1640073811.786920961] [gst_pipeline_node]: simple pipeline camera_input param_cb: unrecognised parameter "video_webrtc.signalling"
[pipeline_node-1] [INFO] [1640073811.794245993] [gst_pipeline_node]: building websockets signalling client for video_webrtc
[pipeline_node-1] [WARN] [1640073811.800858996] [gst_pipeline_node]: simple pipeline camera_input param_cb: unrecognised parameter "video_webrtc.signalling_server"
[pipeline_node-1] [WARN] [1640073811.808529339] [gst_pipeline_node]: simple pipeline camera_input param_cb: unrecognised parameter "video_webrtc.node_id"
[pipeline_node-1] [WARN] [1640073811.815627341] [gst_pipeline_node]: simple pipeline camera_input param_cb: unrecognised parameter "video_webrtc.peer_id"
[pipeline_node-1] [WARN] [1640073811.823224820] [gst_pipeline_node]: simple pipeline camera_input param_cb: unrecognised parameter "video_webrtc.offer"
[pipeline_node-1] [WARN] [1640073811.830472977] [gst_pipeline_node]: simple pipeline camera_input param_cb: unrecognised parameter "video_webrtc.autodial"
[pipeline_node-1] [INFO] [1640073811.844167263] [gst_pipeline_node]: using node_id "9524"
[pipeline_node-1] [INFO] [1640073811.848927562] [gst_pipeline_node]: using peer_id "2504"
[pipeline_node-1] [INFO] [1640073811.853767809] [gst_pipeline_node]: using server "wss://webrtc.nirbheek.in:8443"
[pipeline_node-1] [WARN] [1640073811.861047685] [gst_pipeline_node]: simple pipeline camera_input param_cb: unrecognised parameter "video_webrtc.stun_server"
[pipeline_node-1] [WARN] [1640073811.868562873] [gst_pipeline_node]: simple pipeline camera_input param_cb: unrecognised parameter "video_webrtc.element_name"
[pipeline_node-1] [WARN] [1640073811.875750562] [gst_pipeline_node]: simple pipeline camera_input param_cb: unrecognised parameter "video_webrtc.audio_src_bin_descr"
[pipeline_node-1] [WARN] [1640073811.882966480] [gst_pipeline_node]: simple pipeline camera_input param_cb: unrecognised parameter "video_webrtc.video_src_bin_descr"
[pipeline_node-1] [WARN] [1640073811.890296355] [gst_pipeline_node]: simple pipeline camera_input param_cb: unrecognised parameter "video_webrtc.video_sink_bin_descr"
[pipeline_node-1] [WARN] [1640073811.897929147] [gst_pipeline_node]: simple pipeline camera_input param_cb: unrecognised parameter "video_webrtc.audio_sink_bin_descr"
[pipeline_node-1] [DEBUG] [1640073811.905362772] [gst_pipeline_node]: building initial webrtc pipes
[pipeline_node-1] [DEBUG] [1640073811.910204686] [gst_pipeline_node]: webrtcbin name=sendrecv bundle-policy=max-bundle stun-server=stun://stun.l.google.com:19302  rosimagesrc ros-topic=/my_image ! video/x-raw,width=640,height=360,framerate=1/15,format=BGR,colorimetry=1:1:7:4,interlace-mode=progressive ! videoconvert ! queue ! vp8enc deadline=1 ! rtpvp8pay ! application/x-rtp,media=video,encoding-name=VP8,payload=97 ! queue  ! sendrecv.
[pipeline_node-1] [DEBUG] [1640073811.981418346] [gst_pipeline_node]: initial webrtc pipes built
[pipeline_node-1] [DEBUG] [1640073811.987277861] [gst_pipeline_node]: transport connected callbacks
[pipeline_node-1] [INFO] [1640073811.992883575] [gst_pipeline_node]: adding section video_webrtc
[pipeline_node-1] [INFO] [1640073811.998190435] [gst_pipeline_node]: setting pipeline to PLAYING
[pipeline_node-1] [DEBUG] [1640073812.000015274] [rcl]: Initializing ROS client library, for context at address: 0x24a6f350
[pipeline_node-1] [DEBUG] [1640073812.000201940] [rcl]: Using security: false
[pipeline_node-1] [WARN] [1640073812.026031609] [rclcpp]: logging was initialized more than once
[pipeline_node-1] [INFO] [1640073812.045354317] [gst_image_src_node]: stream_start at 1640073812045346452
[pipeline_node-1] [INFO] [1640073812.045749889] [gst_image_sink_node]: preparing video with caps 'video/x-raw, width=(int)640, height=(int)360, framerate=(fraction)1/15, format=(string)BGR, colorimetry=(string)1:1:7:4, interlace-mode=(string)progressive'
[pipeline_node-1] [INFO] [1640073812.046066815] [gst_image_sink_node]: setcaps format string is BGR 
[pipeline_node-1] [INFO] [1640073812.046176762] [gst_image_sink_node]: setcaps n_components is 3
[pipeline_node-1] [INFO] [1640073812.046246502] [gst_image_sink_node]: setcaps bits is 8
[pipeline_node-1] [INFO] [1640073812.046308637] [gst_image_sink_node]: setcaps pixel_stride is 3
[pipeline_node-1] [INFO] [1640073812.047940247] [gst_image_src_node]: getcaps with filter 'NULL'
[pipeline_node-1] [INFO] [1640073812.048678318] [gst_image_src_node]: waiting for first message
[pipeline_node-1] [ERROR] [1640073812.540753327] [gst_pipeline_node]: remote says "ERROR peer '2504' not found"
[pipeline_node-1] [INFO] [1640073813.686258478] [gst_image_sink_node]: stream_start at 1640073813686236603
[pipeline_node-1] [INFO] [1640073814.077561679] [gst_image_src_node]: preparing video with caps 'video/x-raw, format=(string)BGR, height=(int)360, width=(int)640, framerate=(fraction)1/15, colorimetry=(string)1:1:7:4, interlace-mode=(string)progressive'
[ERROR] [pipeline_node-1]: process has died [pid 5388, exit code -11, cmd '/home/ubuntu/robot/install/gst_pipeline/lib/gst_pipeline/pipeline_node --ros-args --log-level debug --ros-args -r __node:=gst_pipeline_node --params-file /home/ubuntu/robot/install/bootstrap/share/bootstrap/config/rasptank.config.yaml'].

I am not entirely sure about what you mean by ROS message (metadata only). It looks to me that the crash is happening in the GStreamer plugin, what do you think?

The signalling server currently configured is shown below: signalling_server: 'wss://webrtc.nirbheek.in:8443'