Native support for substreams in a single stream

[felipecrs]

I have always had a wish: I want to be able to switch between multiple streams for a single camera in Home Assistant in a better way than through multiple camera entities.

I have been thinking about this since weeks, and I think I came up with some sort of solution. I may be totally wrong though.

First of all, go2rtc to support multiple video tracks in a single stream:

streams:
  my_camera:
    - rtsp://admin:12345@192.168.1.33:554/0/av0
    - rtsp://admin:12345@192.168.1.33:554/0/av1#substream=1

When streaming from go2rtc with rtsp://127.0.0.1:8554/my_camera, the substream will never be used, unless I ask with rtsp://127.0.0.1:8554/my_camera?substream=1.

The number 1 here means the number of the substream, where the main stream is implicitly 0. I believe there should not be a limitation on the amount of substreams for a stream.

However, go2rtc exposes the list of substreams for a given stream through some API.

Then, the WebRTC integration/card would pull this information when starting the camera feed, and if more than one stream/substreams are found, the card renders a button to switch between them.

What do you think?

[AlexxIT]

I do not know the mechanism to safely switch the current stream for WebRTC and for MSE. It could be a different codec. Or a different profile of the same codec. This mechanism needs to be investigated.

The easiest thing to do is to make several streams and a switch in the card. The switch will completely restart the mechanism for requesting a new stream from the beginning.

Different codecs supported in HLS and DASH protocols.

[felipecrs]

The easiest thing to do is to make several streams and a switch in the card.

My main issue with this is that it would require additional configuration for each camera in each card I have. I wanted to avoid it.

The switch will completely restart the mechanism for requesting a new stream from the beginning.

But I agree with this, even for my initial suggestion of having multiple substreams in a single one. At least we could start by that, and if later we can find a way to make the switch happen more in a more graceful way, we do.

I guess "requesting a new stream from the beginning" when switching between substreams also solves the codec differences concerns. If the codecs of the substream aren't supported, the card would just error as it would normally.

[AlexxIT]

Maybe someday I will make an API to change the active stream for the selected client. This can be useful for tablets on the wall. Something similar Birdseye does in Frigate.

[felipecrs]

That's cool! Thanks a lot for your thoughts!

[lindhe]

Sorry for hijacking this thread a bit, but I want to ask about what you show there: having the stream object be a list and selecting the substream using the substream=1 parameter. Does that work as of v0.12.1, or is that something you propose to be implemented?

[felipecrs]

It was a feature suggestion, not something already implemented.

[alve89]

This is what I get displayed when I click on info of camera1:

{
  "producers": [
    {
      "url": "rtsp://user:pass@camera1.lan.k41/Preview_01_main"
    },
    {
      "url": "rtsp://user:pass@camera1.lan.k41/Preview_01_sub"
    }
  ],
  "consumers": []
}

Just for my understanding. Is this thread about switching between multiple stream URLs stored in the producers array? Is it already possible?

[AlexxIT]

This feature implemented in the latest WebRTC integration version

[jcasarini]

Hi! I'm trying to implement this, but I can't find a way of actually using it. I'm running the go2rtc Add-on in Home Assistant, I loaded both streams of each camera in the config.

streams:
    fondo: 
        - rtsp://192.168.18.111:554/user=admin_password=REDACTED_channel=0_stream=0.sdp?real_stream
        - rtsp://192.168.18.111:554/user=admin_password=REDACTED_channel=1_stream=1.sdp?real_stream
    frente: 
        - rtsp://192.168.18.113:554/user=admin_password=REDACTED_channel=0_stream=0.sdp?real_stream
        - rtsp://192.168.18.113:554/user=admin_password=REDACTED_channel=1_stream=1.sdp?real_stream

Now if I go to the info link it shows:

{
  "producers": [
    {
      "url": "rtsp://192.168.18.111:554/user=admin_password=Mo4q8veto_channel=0_stream=0.sdp?real_stream"
    },
    {
      "url": "rtsp://192.168.18.111:554/user=admin_password=Mo4q8veto_channel=1_stream=1.sdp?real_stream"
    }
  ],
  "consumers": []
}

How can I switch between them? Is there anyway of placing a button (service) on the overlay of the WebRTC-card to switch between them? This would very useful, as when I'm on a mobile connection the main FHD stream won't even load.

Thanks!

[felipecrs]

@jcasarini this is a feature proposal. It's not supported yet. :)

[felipecrs]

And maybe it will never be. For now your best option is to use separate streams in go2rtc and switch between them as documented here in the card.

https://github.com/AlexxIT/WebRTC#:~:text=your%20camera%20entity_id-,or,-type%3A%20%27

[jcasarini]

Sorry @felipecrs I misinterpreted this post from @AlexxIT then. Or he is referring to the option you just mentioned?

This feature implemented in the latest WebRTC integration version

[felipecrs]

Yes, he is referring to: https://github.com/AlexxIT/WebRTC#:~:text=your%20camera%20entity_id-,or,-type%3A%20%27

[jcasarini]

Great, thanks @felipecrs. I had read that part of the card's documentation a few days ago, but I thought this was an easier way. Sorry for the misunderstanding!

[AlexxIT]

Yes, I was referring to the streams setting of the WebRTC card

[jcasarini]

Yes, I was referring to the streams setting of the WebRTC card

Thanks @AlexxIT, amazing work.