RTSP Video from PicoRaptor works in QGC 4.0.6 and does not display in QGC 4.0.11

[leepadc]

Expected Behavior

Video from RTSP stream should display within QGroundControl

Current Behavior

Video from RTSP stream does not display

Steps to Reproduce:

Please provide an unambiguous set of steps to reproduce the current behavior

1. Start Aircraft and Video Source, Link to aircraft with ground station, Connect tablet to ground station using WiFi
2. Confirm Video stream is viewable in VLC Player on Android Tablet
3. Shut down VLC Player
4. Open QGC, pointing it to the same RTSP source
5. Observe no video within QGC - system reports "WAITING FOR VIDEO"

System Information

• Operating System: Android 9, though someone else trying with an iOS device has reported to me the same problem
• QGC Version: 4.0.6 / 4.011 - 32 bit
• QGC build: Stable - pulled from App store
• Flight Controller: Pixhawk 2, Blue Cube
• Autopilot (with version): ArduCopter 3.6.12 through 4.0.5

Detailed Description

Tested with multiple copters of identical configuration. Tested with multiple Android devices and another user reports the problem occurs on an IOS device as well. Video worked on multiple devices when using QGC 4.0.6. Upgraded device to QGC 4.0.11 and video no longer works. Original Video is 720p video being generated by FLIR Duo Pro R. Video is routed through Pixy F Gimbal then into PicoRaptor video encoder. URL to connect to PicoRaptor is: "rtsp://192.168.168.168:8557/PSIA/Streaming/channels/2?videoCodecType=H.264" Video appears as expected in VLC player on same device and on another Android device that has not yet been updated to QGC 4.0.11.

Under General Tab, Video Settings are as follows. Video Source: RTSP Video Stream RTSP URL: rtsp://192.168.168.168:8557/PSIA/Streaming/channels/2?videoCodecType=H.264 Aspect Ratio: 1.777777 Disable When Disarmed: NOT checked Low Latency Mode: NOT checked

Under Video Stream Popup Window on Flight Screen: Enable: ON (slid right with green showing) Grid Lines: OFF (slid left with grey showing) Video Fit: Fit Height (though other settings fail as well) File Name: NOT Populated Record Stream: Greyed Out

Log Files and Screenshots

N/A

[jaxxzer]

Can you go to Application Settings -> Console -> Set Logging and enable GStreamerLog, VideoManagerLog, and VideoReceiverLog, then restart the application, and go back to Console and save log to file. Please get a log on 4.0.6 and 4.0.11 while the camera is up and running.

[leepadc]

Logs from 4.0.6 and Daily Build are from my primary operating device. Log from 4.0.11 is from my secondary device which I was using to test whether 4.0.11 would work. The Daily Build appears to fail the same on both devices. I am not wanting to move the primary device with 4.0.6 to 4.0.11 as I am unsure of how to get back to 4.0.6 if I do that. Without it I can not fly.

Video_Debug_QGC_Log_Daily_Build_20-11-08.txt Video_Debug_QGC_Log_4.0.6.txt Video_Debug_QGC_Log_4.0.11.txt

[jaxxzer]

You have entered the URI differently on 4.0.6 and on 4.0.11, please double check your input: 4.0.6 log: "Testing configuration for RTSP Stream: "rtsp://192.168.168.168:8557/PSIA/Streaming/channels/2?videoCodecType=H.264"" 4.0.11 log: "Testing configuration for RTSP Stream: "rtsp://192.168.168.168:8557/psia/streaming/channels/2?videoCodecType=H.264""

Can you enter this uri and let us know if it works? rtsp://wowzaec2demo.streamlock.net/vod/mp4:BigBuckBunny_115k.mov

[leepadc]

Good catch, but unfortunately not the issue. That difference was an attempt by me to see if capitalization had anything to do with it. It does not appear to. I should have change back before posting files so I could show it failing with an identical URI. The file below shows the failure in 4.0.11 with the same URI that is working in 4.0.6... "rtsp://192.168.168.168:8557/PSIA/Streaming/channels/2?videoCodecType=H.264"

Video_Debug_QGC_Log_4.0.11_a.txt

Also, I am able to view the BigBuckBunny video in both QGC 4.0.6 and QGC 4.0.11

[pompecukor]

We have also lost video since update of the daily build. In our case UDP. Works on stable, but broke since update from 4.0.6 daily.

[pompecukor]

Hey @DonLakeFlyer I put some details here, hope that it helps you fix it. https://discuss.ardupilot.org/t/latest-daily-broke-udp-video-working-in-qgc-4-0-6-not-working-in-qgc-4-0-11/65496

[DonLakeFlyer]

@pompecukor Is this only on Android? Does it work on other OS for you?

[DonLakeFlyer]

A diff from v4.0.6. to v4.0.11 shows these difference in Video code:

diff --git a/src/VideoStreaming/VideoReceiver.cc b/src/VideoStreaming/VideoReceiver.cc
index a6a95d934..b12893796 100644
--- a/src/VideoStreaming/VideoReceiver.cc
+++ b/src/VideoStreaming/VideoReceiver.cc
@@ -293,6 +293,58 @@ _padProbe(GstElement* element, GstPad* pad, gpointer user_data)
     return TRUE;
 }

+static gboolean
+_filterParserCaps(GstElement* bin, GstPad* pad, GstElement* element, GstQuery* query, gpointer data)
+{
+    Q_UNUSED(bin)
+    Q_UNUSED(pad)
+    Q_UNUSED(element)
+    Q_UNUSED(data)
+
+    if (GST_QUERY_TYPE(query) != GST_QUERY_CAPS) {
+        return FALSE;
+    }
+
+    GstCaps* srcCaps;
+
+    gst_query_parse_caps(query, &srcCaps);
+
+    if (srcCaps == nullptr || gst_caps_is_any(srcCaps)) {
+        return FALSE;
+    }
+
+    GstCaps* sinkCaps = nullptr;
+
+    GstCaps* filter;
+
+    if (sinkCaps == nullptr && (filter = gst_caps_from_string("video/x-h264")) != nullptr) {
+        if (gst_caps_can_intersect(srcCaps, filter)) {
+            sinkCaps = gst_caps_from_string("video/x-h264,stream-format=avc");
+        }
+
+        gst_caps_unref(filter);
+        filter = nullptr;
+    } else if (sinkCaps == nullptr && (filter = gst_caps_from_string("video/x-h265")) != nullptr) {
+        if (gst_caps_can_intersect(srcCaps, filter)) {
+            sinkCaps = gst_caps_from_string("video/x-h265,stream-format=hvc1");
+        }
+
+        gst_caps_unref(filter);
+        filter = nullptr;
+    }
+
+    if (sinkCaps == nullptr) {
+        return FALSE;
+    }
+
+    gst_query_set_caps_result(query, sinkCaps);
+
+    gst_caps_unref(sinkCaps);
+    sinkCaps = nullptr;
+
+    return TRUE;
+}
+
 GstElement*
 VideoReceiver::_makeSource(const QString& uri)
 {
@@ -301,12 +353,12 @@ VideoReceiver::_makeSource(const QString& uri)
         return nullptr;
     }

-    bool isTaisync  = uri.contains("tsusb://");
-    bool isUdp264   = uri.contains("udp://");
-    bool isRtsp     = uri.contains("rtsp://");
-    bool isUdp265   = uri.contains("udp265://");
-    bool isTcpMPEGTS= uri.contains("tcp://");
-    bool isUdpMPEGTS= uri.contains("mpegts://");
+    bool isTaisync  = uri.contains("tsusb://",  Qt::CaseInsensitive);
+    bool isUdp264   = uri.contains("udp://",    Qt::CaseInsensitive);
+    bool isRtsp     = uri.contains("rtsp://",   Qt::CaseInsensitive);
+    bool isUdp265   = uri.contains("udp265://", Qt::CaseInsensitive);
+    bool isTcpMPEGTS= uri.contains("tcp://",    Qt::CaseInsensitive);
+    bool isUdpMPEGTS= uri.contains("mpegts://", Qt::CaseInsensitive);

     GstElement* source  = nullptr;
     GstElement* buffer  = nullptr;
@@ -369,6 +421,8 @@ VideoReceiver::_makeSource(const QString& uri)
             break;
         }

+        g_signal_connect(parser, "autoplug-query", G_CALLBACK(_filterParserCaps), nullptr);
+
         gst_bin_add_many(GST_BIN(bin), source, parser, nullptr);

         // FIXME: AV: Android does not determine MPEG2-TS via parsebin - have to explicitly state which demux to use
@@ -481,7 +535,7 @@ VideoReceiver::_tcp_timeout()
         //   found to be working, only then we actually start the stream.
         QUrl url(_uri);
         //-- If RTSP and no port is defined, set default RTSP port (554)
-        if(_uri.contains("rtsp://") && url.port() <= 0) {
+        if(_uri.contains("rtsp://", Qt::CaseInsensitive) && url.port() <= 0) {
             url.setPort(554);
         }
         _socket = new QTcpSocket;
@@ -566,7 +620,7 @@ VideoReceiver::start()
         return;
     }

-    bool useTcpConnection = uri.contains("rtsp://") || uri.contains("tcp://");
+    bool useTcpConnection = uri.contains("rtsp://", Qt::CaseInsensitive) || uri.contains("tcp://", Qt::CaseInsensitive);

     if (_videoSink == nullptr) {
         qCWarning(VideoReceiverLog) << "Failed because video sink is not set";

[pompecukor]

@pompecukor Is this only on Android? Does it work on other OS for you?

Hi, unfortunately both Android and Windows are affected. Not just me. All our customers are affected. We were running daily on all, now after update to latest recently they all lost video (telemetry is working). The only way to get video to work again is to install the standard version. I don't know how to roll back to older version of daily.

[DonLakeFlyer]

Does turning on gstreamer logging output anything interesting?

[DonLakeFlyer]

Also can you try selecting different video decoder in daily build to see if that makes a difference?

[pompecukor]

Does turning on gstreamer logging output anything interesting?

Can you guide me on how to turn that on, then I can provide the log. Thank you kindly.

I was not aware that there were different decoders in the build. Where are those?

Unfortunately I have to relay as all of these units have been shipped out to clients. But I will do my best to find out.

[DonLakeFlyer]

I was not aware that there were different decoders in the build. Where are those?

App Settings - General - Fly View - Video Decode Priority

Logging: App Settings - Console - bottom of page

[pompecukor]

Hi, we got back a unit and tested with latest apk and it seems to be fine now. I will verify with the distributed units as well. Not sure when update was done. But I am happy.

[DonLakeFlyer]

Thanks for letting us know

[leepadc]

@DonLakeFlyer - The RTSP problem originally reported is still not working in 4.0.11 or in the Daily Build (Development HEAD:0041f3a23 2021-01-04 10:29:40 -0800 32 bit). I just retested and the feed works fine in 4.0.6 but not in 4.0.11 or the daily build. Log file from the Daily Build is attached Video_Debug_QGC_Log_Daily_Build_21-01-04.txt

[DonLakeFlyer]

@leepadc Can you test whether this is android only?

[DonLakeFlyer]

Also can you take a log which has gstreamer debug logging on as well: ![Uploading Screen Shot 2021-01-05 at 9.49.57 AM.png…]()

[leepadc]

• It did not work on an iOS device running 4.0.10.
• I have never been able to get the stream to work in Windows.
• Attached is a zipped file with the Gstream Debug Logger set to 5 / Debug.

Video_Debug_QGC_Log_Daily_Build_21-01-04_with_gstream_DEBUG.zip

[DonLakeFlyer]

The key log output seems to be this:

[D] at ../gst/gstbus.c:341 - "<bus472> [msg 0x89fe9c68] posting on bus error message: 0x89fe9c68, time 99:99:99.999999999, seq-num 10511, element 'source', GstMessageError, gerror=(GError)NULL, debug=(string)"../gst/rtsp/gstrtspsrc.c\(6317\):\ gst_rtspsrc_setup_auth\ \(\):\ /GstPipeline:receiver/GstBin:sourcebin/GstRTSPSrc:source:\012No\ supported\ authentication\ protocol\ was\ found";"

Which seems to be some sort user/password auth problem. No idea what that really means though. Video folks will have to comment.

[leepadc]

@DonLakeFlyer I was hoping some video folks would chime in, but none yet. I also tried the latest release 4.1.1 and the problem persists. Do you have any suggestions on how to engage the "video folks" you reference above?

[LorenzMeier]

The FLIR duo is not a hardware configuration we can test with. Can you provide an example stream that would allow to reproduce this? E.g. gstreamer command line.

Overall there are too many video configurations to support, so going forward we will be limiting this to a few supported reference profiles - h264 is one of them, so I'm not implying your setup is out of scope.

[leepadc]

A FLIR Duo Pro R is one of many HDMI sources we have used. We've fed in multiple computer outputs at 720 and 1080, a Sony RX100, and a consumer video device (Bluray player) all with the same results. The common element is that they are all being encoded into H.264 using an Airborne Innovations PicoRaptor. All of those sources work to successfully pass video through the PicoRaptor when using 4.0.6 but do not starting at 4.0.11. I should also note that even in 4.0.6 when video is running and the radio signal is spotty or interrupted QGC occasionally completely locks up - no more video and no more telemetry, requiring a restart of the application, at which point everything returns as expected. Based on other testing the lock up is definitely linked to the video stream (more details available if helpful). I am not sure the lock up in 4.0.6 and the failure to work in 4.0.11 are related, but thought I would bring it up just in case smarter people see a connection.

I must confess I am not sure how to collect the example stream you are requesting

[LorenzMeier]

I'm continuing the investigation here: https://github.com/mavlink/qgroundcontrol/issues/9440

[jon0x0]

Again mpeg2ts was working for us in 4.0.6 but is not working in newer versions including the current latest.

Here is a very simple gstreamer pipeline that works with the Picoraptor/Picocamera mpeg2_ts mode. Failing just a reversion of what broke it, would it be possible to get this working as an option in QGroundControl? For us this would fix the problem. Some users that need some newer features of QGroundControl so it would be great if that worked.

Working pipeline: gst-launch-1.0 udpsrc port=4444 ! tsdemux ! h264parse ! avdec_h264 ! autovideosink sync=false

tested with Gstreamer 1.18.1