Profile T support

[NarkyAbyss]

Audio output streaming or audio backchannel is only specified in profile T. Using it with other profiles is not standard. go2RTC can work with profile T

What are the missing features still needed to support profile T?

https://www.onvif.org/wp-content/uploads/2018/09/ONVIF_Profile_T_Specification_v1-0.pdf https://www.onvif.org/wp-content/uploads/2022/04/onvif-profile-feature-overview.pdf

[roleoroleo]

I haven't looked into this topic in depth. So I don't know exactly wath are the missing features. And it's not simple to complete this task because there are not many T profile compatible clients that you can use to test the system. I'll try to take a look at the specs. Have you tested if go2RTC works with the current implementation?

[NarkyAbyss]

yes I have tested.

If I probe the camera in go2rtc, I can see the second audio type; however, play audio in go2RTC doesnt seem to work RTSP Audio is pcm ONVIF audio decoder (backchannel) is AAC

"producers": [
    {
      "id": 647,
      "format_name": "rtsp",
      "protocol": "rtsp+tcp",
      "remote_addr": "<ip>:554",
      "url": "rtsp://<ip>/ch0_0.h264",
      "sdp": "v=0\r\no=- 1721277708285861 1 IN IP4 <ip>\r\ns=Session streamed by \"rRTSPServer\"\r\ni=ch0_0.h264\r\nt=0 0\r\na=tool:LIVE555 Streaming Media v2023.01.19\r\na=type:broadcast\r\na=control:*\r\na=range:npt=now-\r\na=x-qt-text-nam:Session streamed by \"rRTSPServer\"\r\na=x-qt-text-inf:ch0_0.h264\r\nm=video 0 RTP/AVP 96\r\nc=IN IP4 0.0.0.0\r\nb=AS:700\r\na=rtpmap:96 H264/90000\r\na=fmtp:96 packetization-mode=1;profile-level-id=640020;sprop-parameter-sets=Z2QAIKwsqAJACjJuAgICgAAA+gAAJxBC,aO48sA==\r\na=control:track1\r\nm=audio 0 RTP/AVP 97\r\nc=IN IP4 0.0.0.0\r\nb=AS:128\r\na=rtpmap:97 L16/8000\r\na=control:track2\r\nm=audio 0 RTP/AVP 98\r\nc=IN IP4 0.0.0.0\r\nb=AS:8\r\na=rtpmap:98 MPEG4-GENERIC/16000\r\na=fmtp:98 streamtype=5;profile-level-id=1;mode=aac-hbr;sizelength=13;indexlength=3;indexdeltalength=3;config=1408\r\na=sendonly;\r\na=control:track3\r\n",
      "user_agent": "go2rtc/1.9.4",
      "medias": [
        "video, recvonly, H264",
        "audio, recvonly, L16/8000",
        "audio, recvonly, MPEG4-GENERIC/16000"
      ],
      "receivers": [
        {
          "id": 648,
          "codec": {
            "codec_name": "h264",
            "codec_type": "video",
            "level": 32,
            "profile": "High"
          },
          "childs": [
            649
          ]
        },
        {
          "id": 650,
          "codec": {
            "codec_name": "pcm_s16be",
            "codec_type": "audio",
            "sample_rate": 8000
          },
          "childs": [
            651
          ]
        },
        {
          "id": 652,
          "codec": {
            "codec_name": "aac",
            "codec_type": "audio",
            "sample_rate": 16000
          },
          "childs": [
            653
          ]
        }
      ]
    }
  ],

[roleoroleo]

You can choose audio decoder in the ONVIF configuration page, between G711 and AAC (not pcm_s16be). But, in general, this shouldn't be a problem: the onvif client asks to the server which codec it supports ([GetAudioDecoderConfigurationOptions request) and, after the resposne, the onvif client must use a protocol supported by the server. Is strange that go2RTC tries to use PCM without asking to the server. Maybe there is an option in go2RTC to overwrite this setting.

[NarkyAbyss]

The PCM is for RTSP audio

[roleoroleo]

Click "ONVIF configuration". Yes, probably is not the best menu...

[NarkyAbyss]

AAC is the return codec

Here is my config

[roleoroleo]

Try G711. Maybe...

EDIT

Please, explain me how are you testing go2RTC. What's your setup?

[NarkyAbyss]

Im running go2rtc with the following yaml config:

streams:
    onvif1: onvif://<ip>:80

you can access a web server on port :1984 to see logs and test streams. one of the option is play audio over the back channel

I try to use this test file: https://www2.cs.uic.edu/~i101/SoundFiles/StarWars3.wav this file works on yi hack http://<ip>/?page=speak

[roleoroleo]

I tested the backchannel... About G711 there is a typo in the service.sh script, so it doesn't work. While AAC works properly when I test it with Onvif Device Test Tool. But if I try to use go2rtc it doesn't work: go2rtc sends some onvif request and then sends a rtsp DESCRIBE. The cam replies properly but the messages stop here. There are no messages about profile T, so it should work. If you have a working cam, please take a capture with wireshark or tcpdump and send it to me.

When you use AAC you must use an audio file encrypted with aac. The format used in the speak page is not ok.

[NarkyAbyss]

sorry closed by accident.

Ill grab a capture later with this file: https://dl.espressif.com/dl/audio/gs-16b-1c-44100hz.m4a

[roleoroleo]

This is better: https://dl.espressif.com/dl/audio/gs-16b-1c-44100hz.aac