[Config Support]: Audio works in RTSP restream but is not available in Recordings or Events

[amdhunter]

Describe the problem you are having

I have been working on getting audio working in Frigate, and it's working on all my cameras that support it (in Live view), but the audio is not being recorded with events and recordings.

One question for this setup, when using sub streams, should I be using only a single camera name for the main and sub stream audio settings? For instance, below, my doorbell is being set to use aac audio from each stream. Is there a better way to do this?

Version

0.12.1-367D724

Frigate config file

Doorbell:
    ffmpeg:
      hwaccel_args:
      #   - -hwaccel
      #   - vaapi
      #   - -hwaccel_device
      #   - /dev/dri/renderD128
        - -hwaccel_output_format
        - yuv420p
      input_args:
        - -avoid_negative_ts
        - make_zero
      #   - -fflags
      #   - nobuffer
      #   - -flags
      #  - low_delay
      #   - -strict
      #   - experimental
      #   - -fflags
      #   - +genpts+discardcorrupt
      #   - -rw_timeout
      #   - "5000000"
      #   - -use_wallclock_as_timestamps
      #   - "1" 
      inputs:
        - path: rtsp://127.0.0.1:8554/Doorbell-sub
          roles:
            - detect
        - path: rtsp://127.0.0.1:8554/Doorbell
          roles:
            - record
    detect:
      width: 640
      height: 480
      fps: 3
    snapshots:
      enabled: true
      timestamp: true
      bounding_box: true
      retain:
        default: 180
    record:
      enabled: true
      retain:
        days: 14
        mode: motion
      events:
        retain:
          default: 60
    objects:
      track:
      - person
      - car
      - bus
      - dog
    motion:
      mask:
        - 507,148,503,0,640,0,640,144,594,163
...
...
...
go2rtc:
  streams:
    Doorbell:
      - rtsp://u:pw@192.168.200.124:554/h264Preview_01_main
      - "ffmpeg:Doorbell#audio=aac"
    Doorbell-sub:
      - rtsp://u:pw@192.168.200.124:554/h264Preview_01_sub
      - "ffmpeg:Doorbell-sub#audio=aac"

Relevant log output

No relevant log entries.  Seeing no errors in Frigate or Go2RTC logs.

Frigate stats

N/A

Operating system

HassOS

Install method

HassOS Addon

Coral version

Other

Any other information that may be helpful

Using OpenVINO for detection.

[NickM-27]

your hwaccel_args and input_args are wrong and you should really not be using manual args, should be using ffmpeg presets

you also have not enabled audio in recordings https://docs.frigate.video/troubleshooting/faqs#audio-in-recordings

[amdhunter]

Ok, got it. I haven't added any ffmpeg presets as an overall setting because I have different camera types. That's why I have them set on this camera specifically. I read that you are also using a Reolink doorbell, can you share your settings for it? If I remember correctly, I had to set those ffmpeg settings on mine to get it to work with Frigate.

Currently, all I have set for ffmpeg is the following:

ffmpeg:
  hwaccel_args:
    - -hwaccel
    - qsv
    - -qsv_device
    - /dev/dri/renderD128

[NickM-27]

Currently, all I have set for ffmpeg is the following:

yeah, don't use that. I would suggest just using preset-vaapi

this is what I use for my doorbell https://docs.frigate.video/configuration/camera_specific#reolink-cameras

[amdhunter]

I guess this is where I am confused. The RTSP stream from the Reolink works with no issues as is -- what do I gain from moving to the http stream?

Also, what does preset-vaapi do for me? I have the hwaccel_args set based on the docs for using the passed-through Intel iGPU. Do I not need this?

[NickM-27]

I guess this is where I am confused. The RTSP stream from the Reolink works with no issues as is -- what do I gain from moving to the http stream?

if it works then that's great, no reason to use something else. In my case the rtsp stream had lots of corruption and packet loss making it entirely unusable for the long term recording and detection (I only use it for two way talk)

Also, what does preset-vaapi do for me? I have the hwaccel_args set based on the docs for using the passed-through Intel iGPU. Do I not need this?

presets allow frigate to do more optimizations than the manual ffmpeg args you are using, it reduces CPU usage and allows us to make changes in the future without requiring you to change the config at all

[amdhunter]

Understood. I will play around with the HTTP stream, but fail back to RTSP if needed. I just wanted to make sure I wasn't missing functionality, or something else by using RTSP.

I see where the confusion on my part is for the ffmpeg settings. Back "in the day", the Intel iGPU for the generation I am using (7th Gen i7), the docs had what I posted above in them. Now, I see that the docs have changed to use the preset, rather than the manual settings.

So just to make sure I know what I am supposed to change, currently this is the entirety of my ffmpeg settings:

ffmpeg:
  output_args:
    record: preset-record-generic-audio-aac
  hwaccel_args:
    - -hwaccel
    - qsv
    - -qsv_device
    - /dev/dri/renderD128

And I should change it to the following, and remove all the presets from the doorbell camera config completely?

ffmpeg:
  output_args:
    record: preset-record-generic-audio-aac
  hwaccel_args: preset-vaapi

Have I got it right?

[NickM-27]

Have I got it right?

yes

[amdhunter]

Following up on this, worked great. Thank you for the help.