[Config Support]: How to enable hardware acceleration for H.265 decoding on one stream of one camera?

[ehn]

Describe the problem you are having

I have a number of cameras with H.264 streams. In order to use hardware acceleration when decoding the video streams, I have this in my config:

ffmpeg:
  hwaccel_args: preset-intel-qsv-h264

I recently added an Amcrest ASH41-B, which uses H.265 for the high-res main stream and H.264 for the low-res substream. How can I configure Frigate to use hardware acceleration for decoding both streams (and the H.264 streams from all the other cameras)?

I suppose I should put hwaccel_args: preset-intel-qsv-h265 somewhere. But where?

Version

0.12.0-E454DAF

Frigate config file

ffmpeg:
  hwaccel_args: preset-intel-qsv-h264

go2rtc:
  streams:
    entrance:
      - rtsp://admin:redacted@192.168.6.8:554/cam/realmonitor?channel=1&subtype=0&authbasic=64
      - "ffmpeg:entrance#audio=aac#audio=opus"
    entrance_sub:
      - rtsp://admin:redacted@192.168.6.8:554/cam/realmonitor?channel=1&subtype=1&authbasic=64
      - "ffmpeg:entrance_sub#audio=aac#audio=opus"
  webrtc:
    candidates:
      - 192.168.1.1:8555
      - stun:8555

cameras:
  entrance:
    ffmpeg:
      output_args:
        record: preset-record-generic-audio-copy
      inputs:
        - path: rtsp://127.0.0.1:8554/entrance
          input_args: preset-rtsp-restream
          roles:
            - record
        - path: rtsp://127.0.0.1:8554/entrance_sub
          input_args: preset-rtsp-restream
          roles:
            - detect
    detect:
      width: 640
      height: 480

Relevant log output

N/A

Frigate stats

No response

Operating system

Other Linux

Install method

Docker Compose

Coral version

USB

Any other information that may be helpful

No response

[blakeblackshear]

You can set hwaccel args at a global, camera, or input level.

You can see the optional values commented out in the reference config: https://docs.frigate.video/configuration/index

[blakeblackshear]

To clarify, the record role does not require decoding. Only the stream assigned the detect role is decoded and needs hwaccel. The record stream is written directly to disk.

[ehn]

Ah, got it. So in this case, I wouldn't need to change anything unless I want to transcode to H.264. I've been considering trying that because of playback issues with the H.265 stream in Chrome. It works with MSE and JSMpeg but not WebRTC.

If I wanted to do that, that goes in the go2rtc config, right? Is there a way to specify hardware acceleration there, too?

[blakeblackshear]

WebRTC isn't used for playback of recordings, only the live view. If you want to transcode for the live view only, you would do that in go2rtc: https://github.com/AlexxIT/go2rtc/wiki/Hardware-acceleration

[ehn]

Would I have to transcode twice, both the recordings in Frigate and the live view in go2rtc then? Currently, neither viewing the recordings nor the live view using WebRTC works.

Relatedly, I thought Chrome supported H.265 by now? Might there be something else causing these issues?

[NickM-27]

Would I have to transcode twice, both the recordings in Frigate and the live view in go2rtc then?

No, if you create a go2rtc stream which is h264 then you could also use that as the source for Frigate recordings

Relatedly, I thought Chrome supported H.265 by now? Might there be something else causing these issues?

It should but of course H.265 has many different profiles and standards. You can use chrome://media-internals to see a log which explains why playback failed

[ehn]

Would I have to transcode twice, both the recordings in Frigate and the live view in go2rtc then?

No, if you create a go2rtc stream which is h264 then you could also use that as the source for Frigate recordings

Ok, that makes sense. I'll try that if I can't get H.265 playback to work.

Relatedly, I thought Chrome supported H.265 by now? Might there be something else causing these issues?

It should but of course H.265 has many different profiles and standards. You can use chrome://media-internals to see a log which explains why playback failed

When viewing the camera page and WebRTC stream, chrome://media-internals looks like this:

No obvious errors, but the video player is blank:

[NickM-27]

Oh, webrtc does not support h.265 so that is expected. Chrome has different players for files, dash (also HLS), and webrtc. It is the DASH/HLS player that now supports h.265

[ehn]

Thanks for walking me through this!

So for live-streaming H.265, should I use MSE? That seems to work.

When trying to view the clip of an event for this camera, I get the black screen with the X and the spinner and this error message: "The media could not be loaded, either because the server or network failed or because the format is not supported."

In chrome://media-internals, there are these error messages:

"MediaSource endOfStream before demuxer initialization completes (before HAVE_METADATA) is treated as an error. This may also occur as consequence of other MediaSource errors before HAVE_METADATA."
{"code":12,"data":{},"group":"PipelineStatus","message":"","stack":[{"file":"media/filters/chunk_demuxer.cc","line":1364}]}

[NickM-27]

Thanks for walking me through this!

So for live-streaming H.265, should I use MSE? That seems to work.

Yeah I think MSE should be fine, just slightly more latency than webrtc

When trying to view the clip of an event for this camera, I get the black screen with the X and the spinner and this error message: "The media could not be loaded, either because the server or network failed or because the format is not supported."

In chrome://media-internals, there are these error messages:

"MediaSource endOfStream before demuxer initialization completes (before HAVE_METADATA) is treated as an error. This may also occur as consequence of other MediaSource errors before HAVE_METADATA."
{"code":12,"data":{},"group":"PipelineStatus","message":"","stack":[{"file":"media/filters/chunk_demuxer.cc","line":1364}]}

Do you see any errors if you look in the nginx logs?

[ehn]

Yes, in fact, there are errors in the NGINX log:

2023-03-21 13:58:43.585089154  2023/03/21 13:58:43 [error] 302#302: *3229 ngx_child_request_wev_handler: upstream returned a bad status 404 while sending to client, client: 192.168.193.13, server: , request: "GET /vod/event/1679399245.436012-vg3aiu/master.m3u8 HTTP/1.1", host: "192.168.1.1:5000", referrer: "http://192.168.1.1:5000/events?cameras=entrance"
2023-03-21 13:58:43.585134966  192.168.193.13 - - [21/Mar/2023:13:58:43 +0000] "GET /vod/event/1679399245.436012-vg3aiu/master.m3u8 HTTP/1.1" 502 559 "http://192.168.1.1:5000/events?cameras=entrance" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/111.0.0.0 Safari/537.36" "-"
2023-03-21 13:58:43.605498444  2023/03/21 13:58:43 [error] 302#302: *3229 ngx_child_request_wev_handler: upstream returned a bad status 404 while sending to client, client: 192.168.193.13, server: , request: "GET /vod/event/1679399245.436012-vg3aiu/master.m3u8 HTTP/1.1", host: "192.168.1.1:5000", referrer: "http://192.168.1.1:5000/events?cameras=entrance"
2023-03-21 13:58:43.605559153  192.168.193.13 - - [21/Mar/2023:13:58:43 +0000] "GET /vod/event/1679399245.436012-vg3aiu/master.m3u8 HTTP/1.1" 502 559 "http://192.168.1.1:5000/events?cameras=entrance" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/111.0.0.0 Safari/537.36" "-"

[NickM-27]

No errors in frigate logs?

[ehn]

Yes, there, too:

2023-03-21 14:05:10.151707722  [2023-03-21 14:05:10] watchdog.entrance              ERROR   : No new recording segments were created for entrance in the last 120s. restarting the ffmpeg record process...
2023-03-21 14:05:10.152019478  [2023-03-21 14:05:10] watchdog.entrance              INFO    : Terminating the existing ffmpeg process...
2023-03-21 14:05:10.152405424  [2023-03-21 14:05:10] watchdog.entrance              INFO    : Waiting for ffmpeg to exit gracefully...

Looks like there are bigger, underlying problems. 🤔

[NickM-27]

yes it does seem so, what happens if you configure that camera like this

cameras:
  entrance:
    ffmpeg:
      hwaccel_args: preset-vaapi
      output_args:
        record: preset-record-generic-audio-copy
      inputs:
        - path: rtsp://127.0.0.1:8554/entrance
          input_args: preset-rtsp-restream
          roles:
            - record
        - path: rtsp://127.0.0.1:8554/entrance_sub
          input_args: preset-rtsp-restream
          roles:
            - detect
    detect:
      width: 640
      height: 480

[ehn]

Trying that now. Seem to be working? Will go and generate some events.

[ehn]

Playback of clips and recordings now seems to be working. Thanks! Do you know what made the difference?

[NickM-27]

Most likely the hardware acceleration args were silently causing an issue (even though recording doesn't use hwaccel, specifying qsv with the wrong input type causes a problem)

Vaapi has automatic profile selection so it doesn't care if the input is h264 or h265

[ehn]

Any reason I shouldn't set hwaccel_args: preset-vaapi globally instead of preset-intel-qsv-h264?

[NickM-27]

No reason, in my experience vaapi is fine and potentially better than qsv for all streams

[ehn]

Thanks! Will try that.

[github-actions[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[ehn]

Using hwaccel_args: preset-vaapi resolved this for me. From my perspective, we can close this.