Debugging RTSP Further

[patrickhulce]

Hi again :) I'm having trouble debugging an RTSP cam source using the existing documentation. Anything obvious I'm missing? What can I try next to pinpoint the issue?

Steps

• Run the RTSP compatibility check.

What I Expected No obvious errors in the logs and the :888/stream displays the camera video.

What Actually Happens No obvious errors in the logs but the :888/stream displays the stub image only.

Additional Context I've followed the guides at https://docs.savant-ai.io/v0.2.7/recipes/0_rtsp_cam_compatibility.html without any obvious issues. Both ping show no loss and ffplay work flawlessly. Additionally, within the docker container ffmpeg -i rtsp://<URI> -frames:v 1 output.jpg pulls images / video with no problems. I've tried both the ffmpeg.sh and rtsp.sh source adapters without many changes.

Logs

``` receiver-1 | Setting pipeline to PAUSED ... receiver-1 | INFO insight::savant::ffmpeg_src::ffmpeg_src+ffmpegsrc0 > Creating FFMpegSource. receiver-1 | [2024-02-12T16:52:37Z INFO ffmpeg_input] Setting log level to Debug receiver-1 | Pipeline is PREROLLING ... receiver-1 | [tcp @ 0x7f3108003280] No default whitelist set receiver-1 | [tcp @ 0x7f3108003280] Original list of addresses: receiver-1 | [tcp @ 0x7f3108003280] Address IP port PORT receiver-1 | [tcp @ 0x7f3108003280] Interleaved list of addresses: receiver-1 | [tcp @ 0x7f3108003280] Address IP port PORT receiver-1 | [tcp @ 0x7f3108003280] Starting connection attempt to IP port PORT receiver-1 | [tcp @ 0x7f3108003280] Successfully connected to IP port PORT broadcaster-1 | INFO insight::savant::gstreamer::runner > Starting pipeline ``... broadcaster-1 | INFO insight::savant::gstreamer::runner > The pipeline is initialized and ready to process data. Initialization took 0:00:00.001612. receiver-1 | [rtsp @ 0x7f3108001040] SDP: receiver-1 | v=0 receiver-1 | o=- 0 0 IN IP4 127.0.0.1 receiver-1 | s=No Name receiver-1 | c=IN IP4 0.0.0.0 receiver-1 | t=0 0 receiver-1 | m=video 0 RTP/AVP 96 receiver-1 | a=control:rtsp://URI_WAS_HERE receiver-1 | a=rtpmap:96 H264/90000 receiver-1 | a=fmtp:96 packetization-mode=1; profile-level-id=7A0028; sprop-parameter-sets=Z3oAKLzZQHgCJ+JhAAADAAEAAAMAPA8YMZY=,aOvjyyLA receiver-1 | receiver-1 | [rtsp @ 0x7f3108001040] video codec set to: h264 receiver-1 | [rtsp @ 0x7f3108001040] RTP Packetization Mode: 1 receiver-1 | [rtsp @ 0x7f3108001040] RTP Profile IDC: 7a Profile IOP: 0 Level: 28 receiver-1 | [rtsp @ 0x7f3108001040] Extradata set to 0x7f31080042c0 (size: 40) receiver-1 | [rtsp @ 0x7f3108001040] setting jitter buffer size to 0 receiver-1 | [rtsp @ 0x7f3108001040] hello state=0 receiver-1 | [h264 @ 0x7f3108005900] nal_unit_type: 7(SPS), nal_ref_idc: 3 receiver-1 | [h264 @ 0x7f3108005900] nal_unit_type: 8(PPS), nal_ref_idc: 3 broadcaster-1 | INFO insight::savant::gstreamer::runner > Received EOS from pipeline0. receiver-1 | [h264 @ 0x7f3108005900] nal_unit_type: 7(SPS), nal_ref_idc: 3 receiver-1 | [h264 @ 0x7f3108005900] nal_unit_type: 8(PPS), nal_ref_idc: 3 broadcaster-1 | INFO insight::savant::gstreamer::runner > The pipeline is about to stop. Operation took 0:00:00.162509. receiver-1 | [h264 @ 0x7f3108005900] nal_unit_type: 1(Coded slice of a non-IDR picture), nal_ref_idc: 0 receiver-1 | [h264 @ 0x7f3108005900] Format yuv422p chosen by get_format(). receiver-1 | [h264 @ 0x7f3108005900] Reinit context to 1920x1088, pix_fmt: yuv422p receiver-1 | [h264 @ 0x7f3108005900] Frame num gap 15 10 receiver-1 | [h264 @ 0x7f3108005900] Frame num gap 15 11 broadcaster-1 | INFO insight::savant::deepstream::encoding > Encoder for codec 'h264' is available receiver-1 | [h264 @ 0x7f3108005900] Frame num gap 15 12 receiver-1 | [h264 @ 0x7f3108005900] Frame num gap 15 13 receiver-1 | [h264 @ 0x7f3108005900] no picture receiver-1 | [h264 @ 0x7f3108005900] co located POCs unavailable broadcaster-1 | INFO insight::savant::adapters::ao_sink::entrypoint > Started MediaMTX, PID: 32 broadcaster-1 | 2024/02/12 16:52:37 INF MediaMTX / rtsp-simple-server v0.22.1 broadcaster-1 | 2024/02/12 16:52:37 INF [RTSP] listener opened on :554 (TCP), :8000 (UDP/RTP), :8001 (UDP/RTCP) broadcaster-1 | 2024/02/12 16:52:37 INF [RTMP] listener opened on :1935 broadcaster-1 | 2024/02/12 16:52:37 INF [HLS] listener opened on :888 broadcaster-1 | 2024/02/12 16:52:37 INF [WebRTC] listener opened on :8889 (HTTP) broadcaster-1 | INFO insight::savant::adapters::ao_sink::entrypoint > Started Always-On-RTSP, PID: 37 broadcaster-1 | INFO insight::savant::adapters::ao_sink > Source test. Starting Always-On-RTSP sink broadcaster-1 | INFO insight::savant::adapters::ao_sink::pipeline::PipelineThread > Source test. Starting pipeline input-pipeline broadcaster-1 | INFO insight::savant::gstreamer::runner > Starting pipeline ``... broadcaster-1 | INFO insight::savant::utils::zeromq > Starting ZMQ source: socket ipc:///tmp/zmq-sockets/input-video.ipc, type ReceiverSocketTypes.SUB, bind False. broadcaster-1 | INFO insight::savant::gstreamer::runner > The pipeline is initialized and ready to process data. Initialization took 0:00:00.001180. broadcaster-1 | INFO insight::savant::adapters::ao_sink::pipeline::PipelineThread > Source test. Starting pipeline output-pipeline broadcaster-1 | INFO insight::savant::gstreamer::runner > Starting pipeline ``... broadcaster-1 | INFO insight::savant::gstreamer::runner > The pipeline is initialized and ready to process data. Initialization took 0:00:00.004540. broadcaster-1 | 2024/02/12 16:52:37 INF [RTSP] [conn 127.0.0.1:44234] opened broadcaster-1 | INFO insight::savant::always_on_rtsp_frame_processor > Frame resolution is 1920x1080 receiver-1 | [rtsp @ 0x7f3108001040] All info found receiver-1 | [h264 @ 0x7f310801f9c0] nal_unit_type: 7(SPS), nal_ref_idc: 3 receiver-1 | [h264 @ 0x7f310801f9c0] nal_unit_type: 8(PPS), nal_ref_idc: 3 broadcaster-1 | 2024/02/12 16:52:38 INF [RTSP] [session a8bb146f] created by 127.0.0.1:44234 broadcaster-1 | 2024/02/12 16:52:39 INF [RTSP] [session a8bb146f] is publishing to path 'stream', with TCP, 1 track (H264) receiver-1 | INFO insight::savant::ffmpeg_src::ffmpeg_src+ffmpegsrc0 > Setting caps to video/x-h264, stream-format=(string)byte-stream, alignment=(string)au, width=(int)1920, height=(int)1080, framerate=(fraction)30/1. receiver-1 | INFO insight::savant::savant_rs_serializer > Sink caps changed to video/x-h264, stream-format=(string)byte-stream, alignment=(string)au, width=(int)1920, height=(int)1080, framerate=(fraction)30/1, parsed=(boolean)true broadcaster-1 | 2024/02/12 16:52:45 INF [HLS] [muxer stream] created (requested by 172.24.21.8) broadcaster-1 | 2024/02/12 16:52:45 INF [HLS] [muxer stream] is converting into HLS, 1 track (H264) broadcaster-1 | Output test: Processed 1000 frames, 30.00 FPS. broadcaster-1 | Output test: Processed 1000 frames, 30.00 FPS. broadcaster-1 | Output test: Processed 1000 frames, 30.00 FPS. ```

Docker Config

```yaml version: "3.3" services: receiver: image: ghcr.io/insight-platform/savant-adapters-gstreamer:latest restart: unless-stopped volumes: - zmq_sockets:/tmp/zmq-sockets - ./assets/test_data:/stub_imgs environment: - ZMQ_ENDPOINT=pub+bind:ipc:///tmp/zmq-sockets/input-video.ipc - FFMPEG_LOGLEVEL=debug # Unused since switch away from ffmpeg.sh - FFMPEG_PARAMS=rtsp_transport=tcp # Unused since switch away from ffmpeg.sh - RTSP_URI=rtsp://:/stream - SOURCE_ID=test entrypoint: /opt/savant/adapters/gst/sources/rtsp.sh broadcaster: image: ghcr.io/insight-platform/savant-adapters-deepstream:latest restart: unless-stopped ports: - "888:888" # HLS volumes: - zmq_sockets:/tmp/zmq-sockets - ./assets/test_data:/stub_imgs environment: - ZMQ_ENDPOINT=sub+connect:ipc:///tmp/zmq-sockets/input-video.ipc - SOURCE_ID=test - STUB_FILE_LOCATION=/stub_imgs/smpte100_1920x1080.jpeg - DEV_MODE=True command: python -m adapters.ds.sinks.always_on_rtsp deploy: resources: reservations: devices: - driver: nvidia count: 1 capabilities: [gpu] volumes: zmq_sockets: ```

[bwsw]

Hello, @patrickhulce I just checked the RTSP compatibility test.

Enter the dir:

cd samples/rtsp_cam_compatibility_test

Set .env to:

URI=rtsp://hello.savant.video:8554/city-traffic

Edit compose, add port for RTSP (554):

    ports:
      - "888:888"    # HLS
      - "554:554"

Run:

docker compose -f docker-compose.x86.yml up

Open in FFplay: ffplay rtsp://localhost:554/stream

Must display the video. Works in my environment.

[bwsw]

@patrickhulce checked HLS (888) too:

[bwsw]

If it does not work for you, please share what environment you are using.

[bwsw]

Correct log must display the information for both containers:

broadcaster-1  | 2024/02/12 17:22:02 INF [HLS] [muxer stream] is converting into HLS, 1 track (H264)
broadcaster-1  | Output test: Processed 1000 frames, 30.06 FPS.
receiver-1     | Processed 1000 frames, 24.03 FPS.
broadcaster-1  | Input test: Processed 1000 frames, 25.27 FPS.
broadcaster-1  | Output test: Processed 1000 frames, 30.00 FPS.
receiver-1     | Processed 1000 frames, 24.78 FPS.
broadcaster-1  | Input test: Processed 1000 frames, 24.78 FPS.
broadcaster-1  | Output test: Processed 1000 frames, 30.00 FPS.

I do not see your receiver (RTSP adapter) playing something. It's likely either an RTSP stream compatibility problem or some other unknown reason.

If you want us to test your particular stream, you need to contact me on Discord and share the URI. We will check, and if there is a bug, we will fix the adapter if possible.

[patrickhulce]

Understood thanks for the reply! Yes your stream is playing just fine in this environment.

My question is what steps can I take to debug why my RTSP input stream is not working even though it is fine in ffmpeg when savant is not involved. Compared to your example I'm missing logs for input test: Processed 1000 frames and receiver-1 | Setting pipeline to PLAYING ... but everything before that looks the same without any error messages.

[patrickhulce]

you need to contact me on Discord and share the URI. We will check, and if there is a bug, we will fix the adapter if possible.

Thanks I'll join the Discord :)

[bwsw]

@patrickhulce, contact me and share the stream in private; we can check it.