RTSPS Support

[starsoccer]

It seems currently RTSPS isnt supported and was curious what would be needed to add support for this?

[scottlamb]

I'm curious if you have a particular use case in mind. In my experience, most people are using RTSP over a LAN to talk to local cameras, where cryptography is often considered less important, and folks often don't have valid certificates set up to take advantage of it anyway. In general, I don't recommend using RTSP(S) over the Internet, in part because the cameras often are terribly insecure in other ways (backdoors, buffer overflows, etc.) so should be firewalled away.

To answer your question, RTSPS isn't officially standardized. The RTSP/1.0 spec (the one everyone uses) doesn't mention it. The RTSP/2.0 spec does, but no one will ever use RTSP/2.0, so at best that spec is useful as commentary on RTSP/1.0. Informally, RTSPS seems to consist of two pieces:

• layer the TCP connection over a standard TLS implementation. Shouldn't be hard. If we implemented #7, callers could supply their own TLS implementation (e.g. choose between rustls, openssl, native-tls, or something else).
• when using UDP, use SRTP. This is more work, I think. RFC 3711 has a bunch of detail to implement. [edit: also, I think there's a separate standard for encrypting RTP extensions, because they didn't think that through the first time.] The webrtc-rs folks have a webrtc-srtp crate we could look at (we'd have to decide whether it's suitable to use as-is, could be made to suitable for both projects, or is just available to borrow ideas from). Not sure if you intend to use UDP or not.

[starsoccer]

So my personal use case for this is to connect to RTSPS streams from Unifi cameras. The cameras are on the same network but as far as Im aware there is no option for me to use RTSP only RTSPS. Currently I am using ffmpeg to handle the stream and convert the stream to wav format which I then do audio transcription on. Id personally like to remove the need for ffmpeg entirely in the process if possible and just go straight from the stream to audio I can use.

[scottlamb]

Cool project!

Okay, that makes sense to me now. I don't get why Ubiquiti forces TLS with poor cert management on a LAN, but I have some of their access points, and I know the management server does the exact same thing, training everyone to click through certificate warnings. 🙄🤦🏻‍♂️ Retina's philosophy is to work with real cameras, even when they make dumb decisions and/or don't implement standards properly.

[scottlamb]

Oh, can you use TCP (RTSP interleaved channels), or do you need UDP support also? As I mentioned, I expect TCP-only is a lot easier to do.

You could also try going through a simple TLS proxy. Something like stunnel, while not necessarily any better from a polished product perspective than ffmpeg, would let you check if Retina works with these cameras other than the need for TLS support.

[starsoccer]

To be totally honestly Im not sure exactly what I need. Ive never used stunnel before but happy to give that a try if it would help narrow down exactly what is needed, or if there is a way with ffmpeg or ffprobe to figure it out happy to try something with either of those.

[scottlamb]

It's been a while, so I just tried out stunnel to refresh my memory. With a stunnel.conf like this:

foreground = yes
pid =

[ubiquiti]
client = yes
accept = :::8085
connect = 127.0.0.1:8443
verifyChain = no

while running stunnel stunnel.conf, I can do curl http://localhost:8085/ and see that it proxies to https://localhost:8443/ without actually authenticating the server's certificate chain. Looks like none of the cameras I have running right now do RTSPS, but I expect it'd work equally well for that.

[starsoccer]

Sorry for the delay, I wasnt able to get this to work as I ran into issues getting this setup. Mostly due to my lack of stunnel knowledge. I am not totally sure what I did wrong but Im guessing it had to do something with the basic auth, or path that I messed up.

So the UI cameras dont each have their own IP with the RTSP stream(as far as Im aware). The way it works is they all go back to one device and that device is what actually has the stream based on path. On top of that basic auth is also required for the stream to work.

So an example may look like this: host: 192.168.0.1 port: 7441 protocol: rtsps path: some-name-id-here query: enableSrtp username: username password: password

So a full URL like when using ffprobe would look something like this: rtsps://username:password@192.168.0.1:7441/123ABC?enableSrtp

[scottlamb]

Hmm, that enableSrtp looks worrying.

Are you running ffprobe rtsps://username:password@192.168.0.1:7441/123ABC?enableSrtp?

You might check if either of these commands works:

• ffprobe -rtsp_transport tcp rtsps://username:password@192.168.0.1:7441/123ABC?enableSrtp
• ffprobe -rtsp_transport tcp rtsps://username:password@192.168.0.1:7441/123ABC

which would tell ffmpeg to use TCP and hopefully makes it not use SRTP for the data channels (either because ffmpeg is using TCP or by omitting that enableSrtp parameter passed to the camera). Cross your fingers about that last part...

If either of those works, my next step would be to try running stunnel with this stunnel.conf

foreground = yes
pid =

[ubiquiti]
client = yes
accept = :::7554
connect = 192.168.0.1:7441
verifyChain = no

and see if either of these commands works:

• ffprobe -rtsp_transport tcp rtsp://username:password@localhost:7554/123ABC?enableSrtp
• ffprobe -rtsp_transport tcp rtsp://username:password@localhost:7554/123ABC

If so, then I'd also see if Retina over stunnel will work with the same URL. If not, that might tell us we need that SRTP support as well as TLS...

[scottlamb]

Huh. Also, folks on the Internet suggest you can use a url like this: rtsp://username:password@192.168.0.1:7447/123ABC. That is,

• Port 7447 rather than 7441
• rtsp rather than rtsps
• No enableSrtp

That may work with Retina without any code changes or proxy server needed.

[starsoccer]

So I tested all 6 scenarios and all 6 seem to have worked. Just to note all 6 below for clarity sake:

1. ffprobe rtsps://username:password@192.168.0.1:7441/123ABC?enableSrtp
2. ffprobe -rtsp_transport tcp rtsps://username:password@192.168.0.1:7441/123ABC?enableSrt
3. ffprobe -rtsp_transport tcp rtsps://username:password@192.168.0.1:7441/123ABC
4. ffprobe -rtsp_transport tcp rtsp://username:password@localhost:7554/123ABC?enableSrtp
5. ffprobe -rtsp_transport tcp rtsp://username:password@localhost:7554/123ABC
6. ffprobe -rtsp_transport tcprtsp://username:password@192.168.0.1:7447/123ABC

So since it seems 6 worked I am going to work on giving Retina a try with that config and seeing if it works and will report back

[starsoccer]

So I gave this a try using the info call and got the following error

E20221020 10:54:11.451 main rtsp] Fatal: Ok response to DESCRIBE CSeq=1: Unable to parse stream 1: invalid fmtp attribute
raw SDP: "v=0\r
o=- 221 0 IN IP4 192.168.0.1\r
s=68D79AE46148_0\r
u=www.evostream.com\r
e=contact@evostream.com\r
c=IN IP4 192.168.0.1\r
t=0 0\r
a=recvonly\r
a=control:*\r
a=range:npt=now-\r
m=audio 0 RTP/AVP 96\r
a=recvonly\r
a=rtpmap:96 mpeg4-generic/48000/1\r
a=control:trackID=0\r
a=fmtp:96 streamtype=5; profile-level-id=15; mode=AAC-hbr; config=1188; SizeLength=13; IndexLength=3; IndexDeltaLength=3;\r
m=audio 0 RTP/AVP 96\r
a=recvonly\r
a=rtpmap:96 opus/48000/2\r
a=control:trackID=1\r
a=fmtp:96\r
m=video 0 RTP/AVP 97\r
a=recvonly\r
a=control:trackID=2\r
a=rtpmap:97 H264/90000\r
a=fmtp:97 profile-level-id=4d402a; packetization-mode=1; sprop-parameter-sets=Z01AKo2NQDwBE/LgLcBAQFAAAD6AABX5CdoIhGo=,aO44gA==\r
"

conn: 192.168.0.100:57835(me)->192.168.0.1:7447@2022-10-20T10:54:11
msg: 0@2022-10-20T10:54:11

[scottlamb]

Workaround for the invalid fmtp attribute error on the ubiquiti-fmtp branch now. It's not merged to main yet because of some seemingly-unrelated test failure on CI that I haven't figured out.

[starsoccer]

Im not really sure if this worked or not:

W20221020 11:47:36.589 main retina::client::parse] ignoring invalid fmtp attribute value "96"
I20221020 11:47:36.589 main retina::codec] no depacketizer for media/encoding_name audio/opus
Next
You probably wanted at least one of --print-sdp or --print-streams?
I20221020 11:47:36.591 main rtsp] Done

[scottlamb]

The ignoring invalid fmtp attribute value "96" warning is as I intended (no longer a hard error). The info command doesn't do much by itself; that's what the You probably wanted at least one of --print-sdp or --print-streams message is about. You can also switch to the mp4 subcommand to actually save a .mp4 file.

[starsoccer]

So I gave mp4 a try to save it to a file as I assume thats the easiest way to verify if its working properly or not. That returned the following fatal error and invalid mp4 file:

W20221020 12:00:06.861 main retina::client::parse] ignoring invalid fmtp attribute value "96"
I20221020 12:00:06.862 main retina::codec] no depacketizer for media/encoding_name audio/opus
I20221020 12:00:06.862 main rtsp::mp4] Using h264 video stream
I20221020 12:00:06.864 main rtsp::mp4] Using mpeg4-generic audio stream (rfc 6381 codec mp4a.40.2)
E20221020 12:00:06.871 main rtsp] Fatal: Timestamp jumped -1537859991 (-17087.333 sec) from 0 to -1537859991 (mod-2^32: 2757107305), npt -17087.333; policy is to allow 0..10 sec only

conn: 192.168.0.100:56634(me)->192.168.0.1:7447@2022-10-20T12:00:06
stream: TCP, interleaved channel ids 0-1
ssrc: 507f0950
seq: 000008e8
pkt: 1923@2022-10-20T12:00:06

Edit: Just in case its helpful full command II ran below

cargo run mp4 --url rtsp://192.168.0.1:7447/ABC123--username username --password password out.mp4

[scottlamb]

Hmm. Right now Retina expects when you're using multiple streams (video and audio here) that the camera sends a correct rtptime header parameter to align them. Looks like this camera doesn't do that. There are a few variations of how Retina handles the initial timestamp (see here, exposed as the --initial-timestamp commandline flag) but I don't think any of them will do the right thing for this camera. I'll have to rethink that.

But since you only care about audio, you could try adding --no-video, which should help. (Likewise, you can likely get a video stream by adding --no-audio.)

[scottlamb]

Actually, maybe --initial-timestamp=ignore will help. I should also add some debug logging to show what the received rtptime actually is...

[starsoccer]

--initial-timestamp=ignore Worked so I think this solves everything I was looking for. I can now use this instead of ffmpeg. And next I will just need to find a speech to text library to use and then use retina to get the audio from a stream and pass it to that, but thats more of a me problem then a retina one.

Appreciate all of your help with this, I think I just need to wait for the ubiquiti-fmtp change to make it into master

[scottlamb]

Great! Merged now.