search

blakeblackshear / frigate

NVR with realtime local object detection for IP cameras
https://frigate.video
MIT License
19.45k stars 1.79k forks source link

[Support]: Need libmfxgen1 library installed to support Alder Lake CPU QSV transcoding #3170

Closed robdoug89 closed 1 year ago

robdoug89 commented 2 years ago

Describe the problem you are having

I have an Alder Lake gen12 CPU that I'm utilizing QuickSync for hardware transcoding.

In order to use the qsv device, the Docker image needs to have libmfxgen1 installed as per the Intel compatibility matrix

The stock Docker image will fail to initialize the hardware encoder/decoders using qsv if this package is not installed (see general Ubuntu bug)

After adding the[ Intel repository](deb [arch=amd64] https://repositories.intel.com/graphics/ubuntu focal main) and installing libmfxgen1 via apt-get results in the ffmpeg instance being able to correctly utilize qsv.

Version

0.10.1-83481AF

Frigate config file

ffmpeg:
        hwaccel_args:
                - -hwaccel
                - qsv
                - -qsv_device
                - /dev/dri/renderD128
        output_args:
          record: -f segment -segment_time 10 -segment_format mp4 -reset_timestamps 1 -strftime 1 -c:v h264_qsv -c:a copy -an
        input_args:
                - -hide_banner
                - -avoid_negative_ts
                - make_zero
                - -fflags
                - +genpts+discardcorrupt+igndts
                - -rtsp_transport
                - tcp
                - -stimeout
                - 5000000
                - -analyzeduration
                - 2147483647
                - -probesize
                - 2147483647

Relevant log output

Error initializing an internal MFX session: unsupported (-3)

FFprobe output from your camera

.

Frigate stats

No response

Operating system

Other Linux

Install method

Docker Compose

Coral version

M.2

Network connection

Wired

Camera make and model

.

Any other information that may be helpful

No response

stale[bot] commented 2 years ago

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

blakeblackshear commented 2 years ago

Have you tried the latest beta version to see if this is still an issue?

robdoug89 commented 2 years ago

I manually upgraded my kernel to 5.18, upgraded Ubuntu to the latest release (22.04 Jammy Jellyfish), and switched to the latest beta image (https://hub.docker.com/layers/frigate/blakeblackshear/frigate/0.11.0-beta2/images/sha256-ae2cd475eedbf6d72d81069ee304152951af7c865aae2936183036e55912089c?context=explore)

After that I re-enabled QSV hardware encoding to h264 from my HEVC streams and it all appears to be functional!

I've confirmed via intel_gpu_top that the hardware encoder is getting utilized and my CPU utilization is low even after pumping up the output to full 4k encodings.

It's been running for a few hours with no hang ups, but I'll let you know if there are any more crashes.

Thanks!

ozett commented 2 years ago

@robdoug89 how do your following settings from your config help - in contrast to missing in the defaults?

- -analyzeduration
- 2147483647
- -probesize
- 2147483647
charlesmunger commented 2 years ago

@robdoug89 can you share the steps you followed? I am still struggling to get qsv enabled for decode on my streams. I followed your steps - ubuntu 22.04 (clean install), upgrade kernel mainline to 5.18, install intel-media-va-driver-non-free/jammy,now 22.3.1+ds1-1 amd64 [installed].

To start, I verified that everything was working on the host, and hit the ubuntu bug you listed earlier. But once libmfx-gen1.2 was installed, I was happily able to convert an h.264 file recorded via non-hardware-accelerated non-transcoded frigate:

$ ffmpeg -hwaccel qsv -c:v h264_qsv -qsv_device /dev/dri/renderD128 -i recordings/2022-06/20/01/tree_driveway/49.12.mp4 -vf hwdownload,format=nv12 -pix_fmt yuv420p  ~/bar.yuv

And it worked correctly, and used quicksync as verified by load in intel_gpu_top.

However, in the frigate container 0.11.0-beta4 I have no such luck. The same ffmpeg command fails and produces a spew of errors; I've included the relevant bits of the log at trace verbosity.

Logs from command:

[AVHWDeviceContext @ 0x5644ead9ea00] libva: VA-API version 1.11.0
[AVHWDeviceContext @ 0x5644ead9ea00] libva: User requested driver 'iHD'
[AVHWDeviceContext @ 0x5644ead9ea00] libva: Trying to open /usr/lib/jellyfin-ffmpeg/lib/dri/iHD_drv_video.so
[AVHWDeviceContext @ 0x5644ead9ea00] libva: Trying to open /usr/lib/x86_64-linux-gnu/dri/iHD_drv_video.so
[AVHWDeviceContext @ 0x5644ead9ea00] libva: Found init function __vaDriverInit_1_10
[AVHWDeviceContext @ 0x5644ead9ea00] libva: va_openDriver() returns 0
[AVHWDeviceContext @ 0x5644ead9ea00] Initialised VAAPI connection: version 1.11
[AVHWDeviceContext @ 0x5644ead9ea00] VAAPI driver: Intel iHD driver for Intel(R) Gen Graphics - 21.1.1 ().
[AVHWDeviceContext @ 0x5644ead9ea00] Driver not found in known nonstandard list, using standard behaviour.
[AVHWDeviceContext @ 0x5644eae52480] Initialize MFX session: API version is 1.35, implementation version is 1.35
...
[AVHWDeviceContext @ 0x5644eaf00e80] VAAPI driver: Intel iHD driver for Intel(R) Gen Graphics - 21.1.1 ().
[AVHWDeviceContext @ 0x5644eaf00e80] Driver not found in known nonstandard list, using standard behaviour.
[AVHWFramesContext @ 0x5644eaf0fe40] Created surface 0.
[AVHWFramesContext @ 0x5644eaf0fe40] Direct mapping possible.
...
[h264_qsv @ 0x5644ead7bec0] Format qsv chosen by get_format().
[h264_qsv @ 0x5644ead7bec0] Decoder: output is video memory surface

repeated for every sample:

cur_dts is invalid st:0 (0) [init:0 i_done:0 finish:0] (this is harmless if it occurs once at the start per stream)
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x5644ead73fc0] stream 0, sample 1, dts 33000
[h264_qsv @ 0x5644ead7bec0] Error during QSV decoding.: device failed (-17)
Error while decoding stream #0:0: Input/output error

finishing with:

[h264_qsv @ 0x5644ead7bec0] Too many errors when draining, this is a bug. Stop draining and force EOF.
Error while decoding stream #0:0: Internal bug, should not have happened

I tried installing libmfx1, same thing. For science, I tried adding debian sid to apt's sources and installing libmfx-gen1.2 but that did not result in any visible change.

in-container device permission

root@frigate:/media/frigate# ls -la /dev/dri/
total 0
drwxr-xr-x  2 root root        80 Jun 20 04:13 .
drwxr-xr-x 14 root root      4200 Jun 20 04:13 ..
crw-rw----  1 root video 226,   0 Jun 20 04:13 card0
crw-rw----  1 root   109 226, 128 Jun 20 04:13 renderD128
robdoug89 commented 2 years ago

@charlesmunger what does your docker-compose file look like?

I had to add:

environment:
   LIBVA_DRIVERS_PATH: "/usr/lib/x86_64-linux-gnu/dri"
   LIBVA_DRIVER_NAME: "iHD"

to mine before it started working. Though it looks like from the logs the frigate container's ffmpeg is already attempting to look in the right spot

Additionally, I'm still on '0.11.0-beta2' (which is still working for me)

What does your frigate config look like for the ffmpeg section? Under record I'm using:

-f segment -segment_time 10 -segment_format mp4 -reset_timestamps 1 -strftime 1 -c:v h264_qsv -vf scale=3840:2160 -crf 17 -preset medium -movflags +faststart -c:a copy

in your vf section looks like ffmpeg is transferring the stream to/from the QSV chip to system memory. I removed all of that since it didn't appear necessary to move the stream when the encoding was happening on the chip

charlesmunger commented 2 years ago

OK, I've tested all the beta versions of 0.11.0 and... I got beta2 to work with vaapi (working images, intel_gpu_top confirms hwaccel), the other betas (1, 3, 4) fail with a green screen and error logs. Config:

      hwaccel_args: &hwaccel_args
        - -hwaccel
        - vaapi
        - -vaapi_device
        - /dev/dri/renderD128
        - -hwaccel_output_format
        - yuv420p

Same config, everything else identical but running a different beta version:

[2022-06-20 17:09:00] watchdog.tree_driveway         ERROR   : Ffmpeg process crashed unexpectedly for tree_driveway.
[2022-06-20 17:09:00] watchdog.tree_driveway         ERROR   : The following ffmpeg logs include the last 100 lines prior to exit.
[2022-06-20 17:09:00] ffmpeg.tree_driveway.detect    ERROR   : [h264 @ 0x5638405bea80] hardware accelerator failed to decode picture
[2022-06-20 17:09:00] ffmpeg.tree_driveway.detect    ERROR   : Error while decoding stream #0:1: Input/output error
[2022-06-20 17:09:00] ffmpeg.tree_driveway.detect    ERROR   : [h264 @ 0x5638405bea80] Failed to end picture decode issue: 23 (internal decoding error).

So something in beta2 made it work, that then regressed in beta 3.

stale[bot] commented 2 years ago

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

charlesmunger commented 2 years ago

Ok, I got qsv hwaccel "working" on beta7 with this config:

  hwaccel_args:
    - -hwaccel
    - qsv
    - -qsv_device
    - /dev/dri/renderD128
    - -hwaccel_output_format
    - qsv
    - -c:v
    - h264_qsv
    - -gpu_copy
    - "on"
  output_args:
    detect: -vf vpp_qsv=format=yuv420p -f rawvideo -pix_fmt yuv420p

This does result in CPU improvement and I do see activity in intel_gpu_top. I think for Alder Lake on linux, qsv doesn't support yuv420p as a hwdownload format (it does support nv12!), so we need a second pass to convert it to yuv420p in hardware, then use -gpu_copy for accelerated copy from the hardware memory to software. With this set up, each 640x480 7fps reolink detect steam uses 3% of one E-core on my 12700.

I also tried -vf hwdownload,format=nv12,format=yuv420p and it worked, but it it used slightly more CPU than the vpp_qsv configuration, plus I suspect it's less portable since it's not really documented which formats are supported for direct hwdownload with which drivers and hardware.

I think we should consider updating the docs since I suspect that users of that configuration are not actually getting hardware acceleration - if I didn't specify -c:v h264_qsv then I got no errors... but it wasn't actually using hardware acceleration.

I was also tripped up by the yaml converting on to True implicitly, so I had to add those quotes.

jasonmadigan commented 2 years ago

@charlesmunger any other settings you're using to make this work? Upgraded to an Alder Lake CPU recently and have similar to issues. With your settings, I get green screens and errors.

hwaccel_args: -c:v h264_qsv worked in early betas (beta2).

hwaccel_args: -hwaccel vaapi -hwaccel_device /dev/dri/renderD128 -hwaccel_output_format yuv420p works in recent betas, but it weirdly seemed to only use the GPU for a time before dropping back to CPU (and GPU acceleration only came back after a restart). I didn't dig too much into that one though.

charlesmunger commented 2 years ago

That's the full config that works for me. If you run docker logs frigate, what ffmpeg errors do you see with my configuration? And can you share the full ffmpeg command from the debug page (redact any private info from the url)?

jasonmadigan commented 2 years ago

Hmm, not many specific errors (maybe I can increase the verbosity):

[services.d] starting services
[services.d] done.
[2022-07-25 16:39:12] frigate.app                    INFO    : Starting Frigate (0.11.0-37325c7)
Starting migrations
[2022-07-25 16:39:12] peewee_migrate                 INFO    : Starting migrations
There is nothing to migrate
[2022-07-25 16:39:12] peewee_migrate                 INFO    : There is nothing to migrate
[2022-07-25 16:39:12] detector.coral1                INFO    : Starting detection process: 248
[2022-07-25 16:39:12] detector.coral2                INFO    : Starting detection process: 250
[2022-07-25 16:39:12] frigate.app                    INFO    : Output process started: 251
[2022-07-25 16:39:12] ws4py                          INFO    : Using epoll
[2022-07-25 16:39:12] frigate.video                  ERROR   : frontdoor: Unable to read frames from ffmpeg process.
[2022-07-25 16:39:12] frigate.video                  ERROR   : gate: Unable to read frames from ffmpeg process.
[2022-07-25 16:39:12] frigate.video                  ERROR   : pumphouse: Unable to read frames from ffmpeg process.
[2022-07-25 16:39:12] frigate.video                  ERROR   : garage: Unable to read frames from ffmpeg process.
[2022-07-25 16:39:12] frigate.video                  ERROR   : rear: Unable to read frames from ffmpeg process.
[2022-07-25 16:39:12] frigate.video                  ERROR   : patio: Unable to read frames from ffmpeg process.
[2022-07-25 16:39:12] frigate.video                  ERROR   : rear: ffmpeg process is not running. exiting capture thread...
[2022-07-25 16:39:12] frigate.video                  ERROR   : frontlawn: Unable to read frames from ffmpeg process.
[2022-07-25 16:39:12] frigate.video                  ERROR   : garageinterior: Unable to read frames from ffmpeg process.
[2022-07-25 16:39:12] frigate.video                  ERROR   : sidegarden: Unable to read frames from ffmpeg process.
[2022-07-25 16:39:12] frigate.video                  ERROR   : sidegarden: ffmpeg process is not running. exiting capture thread...
[2022-07-25 16:39:12] frigate.edgetpu                INFO    : Attempting to load TPU as usb:0
[2022-07-25 16:39:12] frigate.app                    INFO    : Camera processor started for frontdoor: 259
[2022-07-25 16:39:12] frigate.video                  ERROR   : frontdoor: ffmpeg process is not running. exiting capture thread...
[2022-07-25 16:39:12] frigate.video                  ERROR   : gate: ffmpeg process is not running. exiting capture thread...
[2022-07-25 16:39:15] frigate.edgetpu                INFO    : TPU found
[2022-07-25 16:39:12] frigate.video                  ERROR   : garage: ffmpeg process is not running. exiting capture thread...
[2022-07-25 16:39:12] frigate.video                  ERROR   : pumphouseinside: Unable to read frames from ffmpeg process.
[2022-07-25 16:39:12] frigate.video                  ERROR   : patio: ffmpeg process is not running. exiting capture thread...
[2022-07-25 16:39:12] frigate.video                  ERROR   : frontlawn: ffmpeg process is not running. exiting capture thread...
[2022-07-25 16:39:12] frigate.video                  ERROR   : garageinterior: ffmpeg process is not running. exiting capture thread...
[2022-07-25 16:39:12] frigate.edgetpu                INFO    : Attempting to load TPU as usb:1
[2022-07-25 16:39:12] frigate.video                  ERROR   : pumphouseinside: ffmpeg process is not running. exiting capture thread...
[2022-07-25 16:39:12] frigate.video                  ERROR   : pumphouse: ffmpeg process is not running. exiting capture thread...
[2022-07-25 16:39:12] frigate.app                    INFO    : Camera processor started for gate: 265
[2022-07-25 16:39:15] frigate.edgetpu                INFO    : TPU found
[2022-07-25 16:39:12] frigate.app                    INFO    : Camera processor started for pumphouse: 269
[2022-07-25 16:39:12] frigate.app                    INFO    : Camera processor started for pumphouseinside: 271
[2022-07-25 16:39:12] frigate.app                    INFO    : Camera processor started for rear: 272
[2022-07-25 16:39:12] frigate.app                    INFO    : Camera processor started for garage: 273
[2022-07-25 16:39:12] frigate.app                    INFO    : Camera processor started for patio: 275
[2022-07-25 16:39:12] frigate.app                    INFO    : Camera processor started for frontlawn: 276
[2022-07-25 16:39:12] frigate.app                    INFO    : Camera processor started for garageinterior: 278
[2022-07-25 16:39:12] frigate.app                    INFO    : Camera processor started for sidegarden: 280
[2022-07-25 16:39:12] frigate.app                    INFO    : Capture process started for frontdoor: 281
[2022-07-25 16:39:12] frigate.app                    INFO    : Capture process started for gate: 285
[2022-07-25 16:39:12] frigate.app                    INFO    : Capture process started for pumphouse: 289
[2022-07-25 16:39:12] frigate.app                    INFO    : Capture process started for pumphouseinside: 293
[2022-07-25 16:39:12] frigate.app                    INFO    : Capture process started for rear: 299
[2022-07-25 16:39:12] frigate.app                    INFO    : Capture process started for garage: 303
[2022-07-25 16:39:12] frigate.app                    INFO    : Capture process started for patio: 434
[2022-07-25 16:39:12] frigate.app                    INFO    : Capture process started for frontlawn: 438
[2022-07-25 16:39:12] frigate.app                    INFO    : Capture process started for garageinterior: 484
[2022-07-25 16:39:12] frigate.app                    INFO    : Capture process started for sidegarden: 494
[2022-07-25 16:39:12] ws4py                          INFO    : Using epoll
[2022-07-25 16:39:32] watchdog.frontdoor             ERROR   : Ffmpeg process crashed unexpectedly for frontdoor.
[2022-07-25 16:39:32] watchdog.frontdoor             ERROR   : The following ffmpeg logs include the last 100 lines prior to exit.
[2022-07-25 16:39:32] watchdog.gate                  ERROR   : Ffmpeg process crashed unexpectedly for gate.
[2022-07-25 16:39:32] watchdog.gate                  ERROR   : The following ffmpeg logs include the last 100 lines prior to exit.
[2022-07-25 16:39:32] watchdog.pumphouse             ERROR   : Ffmpeg process crashed unexpectedly for pumphouse.
[2022-07-25 16:39:32] watchdog.pumphouse             ERROR   : The following ffmpeg logs include the last 100 lines prior to exit.
[2022-07-25 16:39:32] watchdog.garage                ERROR   : Ffmpeg process crashed unexpectedly for garage.
[2022-07-25 16:39:32] watchdog.garage                ERROR   : The following ffmpeg logs include the last 100 lines prior to exit.
[2022-07-25 16:39:32] frigate.video                  ERROR   : frontdoor: Unable to read frames from ffmpeg process.
[2022-07-25 16:39:32] frigate.video                  ERROR   : frontdoor: ffmpeg process is not running. exiting capture thread...
[2022-07-25 16:39:32] watchdog.pumphouseinside       ERROR   : Ffmpeg process crashed unexpectedly for pumphouseinside.
[2022-07-25 16:39:32] watchdog.rear                  ERROR   : Ffmpeg process crashed unexpectedly for rear.
[2022-07-25 16:39:32] watchdog.pumphouseinside       ERROR   : The following ffmpeg logs include the last 100 lines prior to exit.
[2022-07-25 16:39:32] watchdog.patio                 ERROR   : Ffmpeg process crashed unexpectedly for patio.
[2022-07-25 16:39:32] watchdog.rear                  ERROR   : The following ffmpeg logs include the last 100 lines prior to exit.
[2022-07-25 16:39:32] watchdog.patio                 ERROR   : The following ffmpeg logs include the last 100 lines prior to exit.
[2022-07-25 16:39:32] watchdog.frontlawn             ERROR   : Ffmpeg process crashed unexpectedly for frontlawn.
[2022-07-25 16:39:32] watchdog.frontlawn             ERROR   : The following ffmpeg logs include the last 100 lines prior to exit.
[2022-07-25 16:39:32] frigate.video                  ERROR   : gate: Unable to read frames from ffmpeg process.
[2022-07-25 16:39:32] frigate.video                  ERROR   : pumphouse: Unable to read frames from ffmpeg process.
[2022-07-25 16:39:32] frigate.video                  ERROR   : gate: ffmpeg process is not running. exiting capture thread...
[2022-07-25 16:39:32] frigate.video                  ERROR   : pumphouse: ffmpeg process is not running. exiting capture thread...
[2022-07-25 16:39:32] frigate.video                  ERROR   : garage: Unable to read frames from ffmpeg process.
[2022-07-25 16:39:32] frigate.video                  ERROR   : garage: ffmpeg process is not running. exiting capture thread...
[2022-07-25 16:39:32] watchdog.garageinterior        ERROR   : Ffmpeg process crashed unexpectedly for garageinterior.
[2022-07-25 16:39:32] watchdog.garageinterior        ERROR   : The following ffmpeg logs include the last 100 lines prior to exit.
[2022-07-25 16:39:32] watchdog.sidegarden            ERROR   : Ffmpeg process crashed unexpectedly for sidegarden.
[2022-07-25 16:39:32] watchdog.sidegarden            ERROR   : The following ffmpeg logs include the last 100 lines prior to exit.
[2022-07-25 16:39:32] frigate.video                  ERROR   : patio: Unable to read frames from ffmpeg process.
[2022-07-25 16:39:32] frigate.video                  ERROR   : pumphouseinside: Unable to read frames from ffmpeg process.
[2022-07-25 16:39:32] frigate.video                  ERROR   : rear: Unable to read frames from ffmpeg process.
[2022-07-25 16:39:32] frigate.video                  ERROR   : patio: ffmpeg process is not running. exiting capture thread...
[2022-07-25 16:39:32] frigate.video                  ERROR   : rear: ffmpeg process is not running. exiting capture thread...
[2022-07-25 16:39:32] frigate.video                  ERROR   : pumphouseinside: ffmpeg process is not running. exiting capture thread...
[2022-07-25 16:39:32] frigate.video                  ERROR   : frontlawn: Unable to read frames from ffmpeg process.
[2022-07-25 16:39:32] frigate.video                  ERROR   : frontlawn: ffmpeg process is not running. exiting capture thread...
[2022-07-25 16:39:32] frigate.video                  ERROR   : sidegarden: Unable to read frames from ffmpeg process.
[2022-07-25 16:39:32] frigate.video                  ERROR   : sidegarden: ffmpeg process is not running. exiting capture thread...
[2022-07-25 16:39:32] frigate.video                  ERROR   : garageinterior: Unable to read frames from ffmpeg process.
[2022-07-25 16:39:32] frigate.video                  ERROR   : garageinterior: ffmpeg process is not running. exiting capture thread...
[2022-07-25 16:39:42] watchdog.frontdoor             ERROR   : Ffmpeg process crashed unexpectedly for frontdoor.
[2022-07-25 16:39:42] watchdog.frontdoor             ERROR   : The following ffmpeg logs include the last 100 lines prior to exit.
[2022-07-25 16:39:42] watchdog.garage                ERROR   : Ffmpeg process crashed unexpectedly for garage.
[2022-07-25 16:39:42] watchdog.garage                ERROR   : The following ffmpeg logs include the last 100 lines prior to exit.
[2022-07-25 16:39:42] frigate.video                  ERROR   : frontdoor: Unable to read frames from ffmpeg process.
[2022-07-25 16:39:42] watchdog.gate                  ERROR   : Ffmpeg process crashed unexpectedly for gate.
[2022-07-25 16:39:42] watchdog.pumphouse             ERROR   : Ffmpeg process crashed unexpectedly for pumphouse.
[2022-07-25 16:39:42] frigate.video                  ERROR   : frontdoor: ffmpeg process is not running. exiting capture thread...
[2022-07-25 16:39:42] watchdog.pumphouse             ERROR   : The following ffmpeg logs include the last 100 lines prior to exit.
[2022-07-25 16:39:42] watchdog.gate                  ERROR   : The following ffmpeg logs include the last 100 lines prior to exit.
[2022-07-25 16:39:42] watchdog.pumphouseinside       ERROR   : Ffmpeg process crashed unexpectedly for pumphouseinside.
[2022-07-25 16:39:42] watchdog.rear                  ERROR   : Ffmpeg process crashed unexpectedly for rear.
[2022-07-25 16:39:42] watchdog.patio                 ERROR   : Ffmpeg process crashed unexpectedly for patio.
[2022-07-25 16:39:42] watchdog.frontlawn             ERROR   : Ffmpeg process crashed unexpectedly for frontlawn.
[2022-07-25 16:39:42] watchdog.pumphouseinside       ERROR   : The following ffmpeg logs include the last 100 lines prior to exit.
[2022-07-25 16:39:42] watchdog.rear                  ERROR   : The following ffmpeg logs include the last 100 lines prior to exit.
[2022-07-25 16:39:42] watchdog.patio                 ERROR   : The following ffmpeg logs include the last 100 lines prior to exit.
[2022-07-25 16:39:42] watchdog.frontlawn             ERROR   : The following ffmpeg logs include the last 100 lines prior to exit.
[2022-07-25 16:39:42] frigate.video                  ERROR   : gate: Unable to read frames from ffmpeg process.
[2022-07-25 16:39:42] frigate.video                  ERROR   : pumphouse: Unable to read frames from ffmpeg process.
[2022-07-25 16:39:42] frigate.video                  ERROR   : garage: Unable to read frames from ffmpeg process.
[2022-07-25 16:39:42] frigate.video                  ERROR   : gate: ffmpeg process is not running. exiting capture thread...
[2022-07-25 16:39:42] frigate.video                  ERROR   : garage: ffmpeg process is not running. exiting capture thread...
[2022-07-25 16:39:42] frigate.video                  ERROR   : pumphouse: ffmpeg process is not running. exiting capture thread...
[2022-07-25 16:39:42] watchdog.sidegarden            ERROR   : Ffmpeg process crashed unexpectedly for sidegarden.
[2022-07-25 16:39:42] watchdog.garageinterior        ERROR   : Ffmpeg process crashed unexpectedly for garageinterior.
[2022-07-25 16:39:42] watchdog.sidegarden            ERROR   : The following ffmpeg logs include the last 100 lines prior to exit.
[2022-07-25 16:39:42] watchdog.garageinterior        ERROR   : The following ffmpeg logs include the last 100 lines prior to exit.
[2022-07-25 16:39:42] frigate.video                  ERROR   : patio: Unable to read frames from ffmpeg process.
[2022-07-25 16:39:42] frigate.video                  ERROR   : patio: ffmpeg process is not running. exiting capture thread...
[2022-07-25 16:39:42] frigate.video                  ERROR   : frontlawn: Unable to read frames from ffmpeg process.
[2022-07-25 16:39:42] frigate.video                  ERROR   : frontlawn: ffmpeg process is not running. exiting capture thread...
[2022-07-25 16:39:42] frigate.video                  ERROR   : pumphouseinside: Unable to read frames from ffmpeg process.
[2022-07-25 16:39:42] frigate.video                  ERROR   : pumphouseinside: ffmpeg process is not running. exiting capture thread...
[2022-07-25 16:39:42] frigate.video                  ERROR   : rear: Unable to read frames from ffmpeg process.
[2022-07-25 16:39:42] frigate.video                  ERROR   : rear: ffmpeg process is not running. exiting capture thread...
[2022-07-25 16:39:42] frigate.video                  ERROR   : garageinterior: Unable to read frames from ffmpeg process.
[2022-07-25 16:39:42] frigate.video                  ERROR   : sidegarden: Unable to read frames from ffmpeg process.
[2022-07-25 16:39:42] frigate.video                  ERROR   : garageinterior: ffmpeg process is not running. exiting capture thread...
[2022-07-25 16:39:42] frigate.video                  ERROR   : sidegarden: ffmpeg process is not running. exiting capture thread...
[2022-07-25 16:39:52] watchdog.frontdoor             ERROR   : Ffmpeg process crashed unexpectedly for frontdoor.
[2022-07-25 16:39:52] watchdog.frontdoor             ERROR   : The following ffmpeg logs include the last 100 lines prior to exit.
[2022-07-25 16:39:52] watchdog.pumphouse             ERROR   : Ffmpeg process crashed unexpectedly for pumphouse.
[2022-07-25 16:39:52] watchdog.gate                  ERROR   : Ffmpeg process crashed unexpectedly for gate.
[2022-07-25 16:39:52] watchdog.pumphouse             ERROR   : The following ffmpeg logs include the last 100 lines prior to exit.
[2022-07-25 16:39:52] watchdog.gate                  ERROR   : The following ffmpeg logs include the last 100 lines prior to exit.
[2022-07-25 16:39:52] watchdog.garage                ERROR   : Ffmpeg process crashed unexpectedly for garage.
[2022-07-25 16:39:52] frigate.video                  ERROR   : frontdoor: Unable to read frames from ffmpeg process.
[2022-07-25 16:39:52] frigate.video                  ERROR   : frontdoor: ffmpeg process is not running. exiting capture thread...
[2022-07-25 16:39:52] watchdog.garage                ERROR   : The following ffmpeg logs include the last 100 lines prior to exit.
[2022-07-25 16:39:52] watchdog.patio                 ERROR   : Ffmpeg process crashed unexpectedly for patio.
[2022-07-25 16:39:52] watchdog.frontlawn             ERROR   : Ffmpeg process crashed unexpectedly for frontlawn.
[2022-07-25 16:39:52] watchdog.patio                 ERROR   : The following ffmpeg logs include the last 100 lines prior to exit.
[2022-07-25 16:39:52] watchdog.rear                  ERROR   : Ffmpeg process crashed unexpectedly for rear.
[2022-07-25 16:39:52] watchdog.pumphouseinside       ERROR   : Ffmpeg process crashed unexpectedly for pumphouseinside.
[2022-07-25 16:39:52] watchdog.frontlawn             ERROR   : The following ffmpeg logs include the last 100 lines prior to exit.
[2022-07-25 16:39:52] watchdog.rear                  ERROR   : The following ffmpeg logs include the last 100 lines prior to exit.
[2022-07-25 16:39:52] watchdog.pumphouseinside       ERROR   : The following ffmpeg logs include the last 100 lines prior to exit.
[2022-07-25 16:39:52] frigate.video                  ERROR   : pumphouse: Unable to read frames from ffmpeg process.
[2022-07-25 16:39:52] frigate.video                  ERROR   : pumphouse: ffmpeg process is not running. exiting capture thread...
[2022-07-25 16:39:52] frigate.video                  ERROR   : gate: Unable to read frames from ffmpeg process.
[2022-07-25 16:39:52] frigate.video                  ERROR   : garage: Unable to read frames from ffmpeg process.
[2022-07-25 16:39:52] frigate.video                  ERROR   : gate: ffmpeg process is not running. exiting capture thread...
[2022-07-25 16:39:52] frigate.video                  ERROR   : garage: ffmpeg process is not running. exiting capture thread...
[2022-07-25 16:39:52] watchdog.sidegarden            ERROR   : Ffmpeg process crashed unexpectedly for sidegarden.
[2022-07-25 16:39:52] watchdog.garageinterior        ERROR   : Ffmpeg process crashed unexpectedly for garageinterior.
[2022-07-25 16:39:52] watchdog.sidegarden            ERROR   : The following ffmpeg logs include the last 100 lines prior to exit.
[2022-07-25 16:39:52] watchdog.garageinterior        ERROR   : The following ffmpeg logs include the last 100 lines prior to exit.
[2022-07-25 16:39:52] frigate.video                  ERROR   : rear: Unable to read frames from ffmpeg process.
[2022-07-25 16:39:52] frigate.video                  ERROR   : patio: Unable to read frames from ffmpeg process.
[2022-07-25 16:39:52] frigate.video                  ERROR   : rear: ffmpeg process is not running. exiting capture thread...
[2022-07-25 16:39:52] frigate.video                  ERROR   : patio: ffmpeg process is not running. exiting capture thread...
[2022-07-25 16:39:52] frigate.video                  ERROR   : frontlawn: Unable to read frames from ffmpeg process.
[2022-07-25 16:39:52] frigate.video                  ERROR   : frontlawn: ffmpeg process is not running. exiting capture thread...
[2022-07-25 16:39:52] frigate.video                  ERROR   : pumphouseinside: Unable to read frames from ffmpeg process.
[2022-07-25 16:39:52] frigate.video                  ERROR   : pumphouseinside: ffmpeg process is not running. exiting capture thread...
[2022-07-25 16:39:52] frigate.video                  ERROR   : garageinterior: Unable to read frames from ffmpeg process.
[2022-07-25 16:39:52] frigate.video                  ERROR   : sidegarden: Unable to read frames from ffmpeg process.
[2022-07-25 16:39:52] frigate.video                  ERROR   : garageinterior: ffmpeg process is not running. exiting capture thread...
[2022-07-25 16:39:52] frigate.video                  ERROR   : sidegarden: ffmpeg process is not running. exiting capture thread...
[2022-07-25 16:40:02] watchdog.frontdoor             ERROR   : Ffmpeg process crashed unexpectedly for frontdoor.
[2022-07-25 16:40:02] watchdog.frontdoor             ERROR   : The following ffmpeg logs include the last 100 lines prior to exit.
[2022-07-25 16:40:02] watchdog.pumphouse             ERROR   : Ffmpeg process crashed unexpectedly for pumphouse.
[2022-07-25 16:40:02] watchdog.pumphouse             ERROR   : The following ffmpeg logs include the last 100 lines prior to exit.
[2022-07-25 16:40:02] watchdog.garage                ERROR   : Ffmpeg process crashed unexpectedly for garage.
[2022-07-25 16:40:02] watchdog.garage                ERROR   : The following ffmpeg logs include the last 100 lines prior to exit.
[2022-07-25 16:40:02] watchdog.gate                  ERROR   : Ffmpeg process crashed unexpectedly for gate.
[2022-07-25 16:40:02] watchdog.gate                  ERROR   : The following ffmpeg logs include the last 100 lines prior to exit.
[2022-07-25 16:40:02] frigate.video                  ERROR   : frontdoor: Unable to read frames from ffmpeg process.
[2022-07-25 16:40:02] frigate.video                  ERROR   : frontdoor: ffmpeg process is not running. exiting capture thread...
[2022-07-25 16:40:02] watchdog.rear                  ERROR   : Ffmpeg process crashed unexpectedly for rear.
[2022-07-25 16:40:02] watchdog.pumphouseinside       ERROR   : Ffmpeg process crashed unexpectedly for pumphouseinside.
[2022-07-25 16:40:02] watchdog.frontlawn             ERROR   : Ffmpeg process crashed unexpectedly for frontlawn.
[2022-07-25 16:40:02] watchdog.rear                  ERROR   : The following ffmpeg logs include the last 100 lines prior to exit.
[2022-07-25 16:40:02] watchdog.patio                 ERROR   : Ffmpeg process crashed unexpectedly for patio.
[2022-07-25 16:40:02] frigate.video                  ERROR   : gate: Unable to read frames from ffmpeg process.
[2022-07-25 16:40:02] frigate.video                  ERROR   : garage: Unable to read frames from ffmpeg process.
[2022-07-25 16:40:02] frigate.video                  ERROR   : pumphouse: Unable to read frames from ffmpeg process.
[2022-07-25 16:40:02] watchdog.pumphouseinside       ERROR   : The following ffmpeg logs include the last 100 lines prior to exit.
[2022-07-25 16:40:02] watchdog.frontlawn             ERROR   : The following ffmpeg logs include the last 100 lines prior to exit.
[2022-07-25 16:40:02] watchdog.patio                 ERROR   : The following ffmpeg logs include the last 100 lines prior to exit.
[2022-07-25 16:40:02] frigate.video                  ERROR   : garage: ffmpeg process is not running. exiting capture thread...
[2022-07-25 16:40:02] frigate.video                  ERROR   : pumphouse: ffmpeg process is not running. exiting capture thread...
[2022-07-25 16:40:02] frigate.video                  ERROR   : gate: ffmpeg process is not running. exiting capture thread...
[2022-07-25 16:40:02] watchdog.sidegarden            ERROR   : Ffmpeg process crashed unexpectedly for sidegarden.
[2022-07-25 16:40:02] watchdog.sidegarden            ERROR   : The following ffmpeg logs include the last 100 lines prior to exit.
[2022-07-25 16:40:02] watchdog.garageinterior        ERROR   : Ffmpeg process crashed unexpectedly for garageinterior.
[2022-07-25 16:40:02] watchdog.garageinterior        ERROR   : The following ffmpeg logs include the last 100 lines prior to exit.
[2022-07-25 16:40:02] frigate.video                  ERROR   : pumphouseinside: Unable to read frames from ffmpeg process.
[2022-07-25 16:40:02] frigate.video                  ERROR   : patio: Unable to read frames from ffmpeg process.
[2022-07-25 16:40:02] frigate.video                  ERROR   : frontlawn: Unable to read frames from ffmpeg process.
[2022-07-25 16:40:02] frigate.video                  ERROR   : patio: ffmpeg process is not running. exiting capture thread...
[2022-07-25 16:40:02] frigate.video                  ERROR   : pumphouseinside: ffmpeg process is not running. exiting capture thread...
[2022-07-25 16:40:02] frigate.video                  ERROR   : rear: Unable to read frames from ffmpeg process.
[2022-07-25 16:40:02] frigate.video                  ERROR   : rear: ffmpeg process is not running. exiting capture thread...
[2022-07-25 16:40:02] frigate.video                  ERROR   : frontlawn: ffmpeg process is not running. exiting capture thread...
[2022-07-25 16:40:02] frigate.video                  ERROR   : sidegarden: Unable to read frames from ffmpeg process.
[2022-07-25 16:40:02] frigate.video                  ERROR   : garageinterior: Unable to read frames from ffmpeg process.
[2022-07-25 16:40:02] frigate.video                  ERROR   : sidegarden: ffmpeg process is not running. exiting capture thread...
[2022-07-25 16:40:02] frigate.video                  ERROR   : garageinterior: ffmpeg process is not running. exiting capture thread...
[2022-07-25 16:40:11] ws4py                          INFO    : Managing websocket [Local => 127.0.0.1:5002 | Remote => 127.0.0.1:32826]
[2022-07-25 16:40:11] ws4py                          INFO    : Managing websocket [Local => 127.0.0.1:8082 | Remote => 127.0.0.1:52770]
[2022-07-25 16:40:12] watchdog.frontdoor             ERROR   : Ffmpeg process crashed unexpectedly for frontdoor.
[2022-07-25 16:40:12] watchdog.frontdoor             ERROR   : The following ffmpeg logs include the last 100 lines prior to exit.
[2022-07-25 16:40:12] watchdog.garage                ERROR   : Ffmpeg process crashed unexpectedly for garage.
[2022-07-25 16:40:12] watchdog.garage                ERROR   : The following ffmpeg logs include the last 100 lines prior to exit.
[2022-07-25 16:40:12] watchdog.gate                  ERROR   : Ffmpeg process crashed unexpectedly for gate.
[2022-07-25 16:40:12] watchdog.gate                  ERROR   : The following ffmpeg logs include the last 100 lines prior to exit.
[2022-07-25 16:40:12] watchdog.pumphouse             ERROR   : Ffmpeg process crashed unexpectedly for pumphouse.
[2022-07-25 16:40:12] watchdog.pumphouse             ERROR   : The following ffmpeg logs include the last 100 lines prior to exit.
[2022-07-25 16:40:12] frigate.video                  ERROR   : frontdoor: Unable to read frames from ffmpeg process.
[2022-07-25 16:40:12] frigate.video                  ERROR   : frontdoor: ffmpeg process is not running. exiting capture thread...
[2022-07-25 16:40:12] watchdog.rear                  ERROR   : Ffmpeg process crashed unexpectedly for rear.
[2022-07-25 16:40:12] watchdog.frontlawn             ERROR   : Ffmpeg process crashed unexpectedly for frontlawn.
[2022-07-25 16:40:12] watchdog.rear                  ERROR   : The following ffmpeg logs include the last 100 lines prior to exit.
[2022-07-25 16:40:12] watchdog.patio                 ERROR   : Ffmpeg process crashed unexpectedly for patio.
[2022-07-25 16:40:12] watchdog.pumphouseinside       ERROR   : Ffmpeg process crashed unexpectedly for pumphouseinside.
[2022-07-25 16:40:12] frigate.video                  ERROR   : garage: Unable to read frames from ffmpeg process.
[2022-07-25 16:40:12] watchdog.frontlawn             ERROR   : The following ffmpeg logs include the last 100 lines prior to exit.
[2022-07-25 16:40:12] watchdog.patio                 ERROR   : The following ffmpeg logs include the last 100 lines prior to exit.
[2022-07-25 16:40:12] frigate.video                  ERROR   : pumphouse: Unable to read frames from ffmpeg process.
[2022-07-25 16:40:12] frigate.video                  ERROR   : gate: Unable to read frames from ffmpeg process.
[2022-07-25 16:40:12] frigate.video                  ERROR   : garage: ffmpeg process is not running. exiting capture thread...
[2022-07-25 16:40:12] watchdog.pumphouseinside       ERROR   : The following ffmpeg logs include the last 100 lines prior to exit.
[2022-07-25 16:40:12] frigate.video                  ERROR   : pumphouse: ffmpeg process is not running. exiting capture thread...
[2022-07-25 16:40:12] frigate.video                  ERROR   : gate: ffmpeg process is not running. exiting capture thread...
[2022-07-25 16:40:12] watchdog.sidegarden            ERROR   : Ffmpeg process crashed unexpectedly for sidegarden.
[2022-07-25 16:40:12] watchdog.sidegarden            ERROR   : The following ffmpeg logs include the last 100 lines prior to exit.
[2022-07-25 16:40:12] watchdog.garageinterior        ERROR   : Ffmpeg process crashed unexpectedly for garageinterior.
[2022-07-25 16:40:12] watchdog.garageinterior        ERROR   : The following ffmpeg logs include the last 100 lines prior to exit.
[2022-07-25 16:40:12] frigate.video                  ERROR   : pumphouseinside: Unable to read frames from ffmpeg process.
[2022-07-25 16:40:12] frigate.video                  ERROR   : pumphouseinside: ffmpeg process is not running. exiting capture thread...
[2022-07-25 16:40:12] frigate.video                  ERROR   : frontlawn: Unable to read frames from ffmpeg process.
[2022-07-25 16:40:12] frigate.video                  ERROR   : frontlawn: ffmpeg process is not running. exiting capture thread...
[2022-07-25 16:40:12] frigate.video                  ERROR   : rear: Unable to read frames from ffmpeg process.
[2022-07-25 16:40:12] frigate.video                  ERROR   : patio: Unable to read frames from ffmpeg process.
[2022-07-25 16:40:12] frigate.video                  ERROR   : rear: ffmpeg process is not running. exiting capture thread...
[2022-07-25 16:40:12] frigate.video                  ERROR   : patio: ffmpeg process is not running. exiting capture thread...
[2022-07-25 16:40:12] frigate.video                  ERROR   : garageinterior: Unable to read frames from ffmpeg process.
[2022-07-25 16:40:12] frigate.video                  ERROR   : garageinterior: ffmpeg process is not running. exiting capture thread...
[2022-07-25 16:40:12] frigate.video                  ERROR   : sidegarden: Unable to read frames from ffmpeg process.
[2022-07-25 16:40:12] frigate.video                  ERROR   : sidegarden: ffmpeg process is not running. exiting capture thread...
[2022-07-25 16:40:14] ws4py                          INFO    : Terminating websocket [Local => 127.0.0.1:8082 | Remote => 127.0.0.1:52770]
[2022-07-25 16:40:14] ws4py                          INFO    : Terminating websocket [Local => 127.0.0.1:5002 | Remote => 127.0.0.1:32826]
[2022-07-25 16:40:14] ws4py                          INFO    : Managing websocket [Local => 127.0.0.1:5002 | Remote => 127.0.0.1:32830]
jasonmadigan commented 2 years ago

thanks @charlesmunger, debug config: https://gist.github.com/jasonmadigan/75dfadef3703a3a13ef732bd4788539a

charlesmunger commented 2 years ago

@jasonmadigan That config is still using vaapi, not the qsv config in https://github.com/blakeblackshear/frigate/issues/3170#issuecomment-1192046751

jasonmadigan commented 2 years ago

@charlesmunger sorry, switched back to the old config to get things up and running again. Debug with the QSV settings (and green cameras):

https://gist.github.com/jasonmadigan/75dfadef3703a3a13ef732bd4788539a

Log output is much the same, no real info in the errors from ffmpeg.

jasonmadigan commented 2 years ago

Update: nevermind, think I figured it out. Unraid currently (6.10.3) uses kernel 5.15.46. Knew it had a number of issues with the iGPU from various threads. Updated to a test build of Unraid, 6.11.0-beta9, which includes a new 5.18.12 kernel. QSV works again, with your config from above (and can also see load again via the GPU). Thanks @charlesmunger

charlesmunger commented 2 years ago

That's great to hear! One more suggestion though - you don't appear to be using substreams for your cams. If you did, it'd mean less work to decode. You could:

  1. Switch to separate streams for record/detect - higher fps on record for better motion in clips, lower res on detect for better CPU performance.
  2. If for some reason you want to just use the one stream, you can do your downscaling in hardware before passing it on to frigate's motion detection. Instead of -vf vpp_qsv=format=yuv420p try passing -vf scale_qsv=w=1280:h=720,vpp_qsv=format=yuv420p (not actually tested by me!).
charlesmunger commented 2 years ago

For a docs update, sounds like we need:

  1. Tell alder lake users to use kernel 5.18 or newer
  2. Add -hwaccel_output_format qsv to the existing hwaccel_args example
  3. Add -c:v h264_qsv -gpu_copy "on" or the h265 equivalent to input_args. This belongs in input_args so that users with a mix of h264 and h265 cameras will get correct behavior.
  4. Add -vf vpp_qsv=format=yuv420p to output args.

Unfortunately this involves overriding a big chunk of frigate's built in ffmpeg args creation.

My final configuration:

  input_args:
    - -c:v
    - h264_qsv
    - -gpu_copy
    - "on"
  hwaccel_args:
    - -hwaccel
    - qsv
    - -qsv_device
    - /dev/dri/renderD128
    - -hwaccel_output_format
    - qsv 
  output_args:
    detect: -vf vpp_qsv=format=yuv420p -f rawvideo -pix_fmt yuv420p

I'll take a stab at a docs update PR.

NickM-27 commented 2 years ago

For a docs update, sounds like we need:

Seems this would be an additional section just for 12th gen, haven't seen previous gen (even 11th) having to use these args

Feel free to do this or perhaps I'll work on something myself, but I think we might want to re-organize these args. Especially as we are getting more and more variations figured out per generation of intel and encoding, I think a table or some other visual organization would help so it isn't a long list of potential exceptions.

NickM-27 commented 2 years ago

Update: nevermind, think I figured it out. Unraid currently (6.10.3) uses kernel 5.15.46. Knew it had a number of issues with the iGPU from various threads. Updated to a test build of Unraid, 6.11.0-beta9, which includes a new 5.18.12 kernel. QSV works again, with your config from above (and can also see load again via the GPU). Thanks @charlesmunger

Where does one find information on Unraid 6.11 beta? haven't seen anything on it

jasonmadigan commented 2 years ago

Update: nevermind, think I figured it out. Unraid currently (6.10.3) uses kernel 5.15.46. Knew it had a number of issues with the iGPU from various threads. Updated to a test build of Unraid, 6.11.0-beta9, which includes a new 5.18.12 kernel. QSV works again, with your config from above (and can also see load again via the GPU). Thanks @charlesmunger

Where does one find information on Unraid 6.11 beta? haven't seen anything on it

It's in test, Unraid don't seem to publicise these as much as the RCs, but there were some links shared on their forums by someone.

Install Plugin > https://s3.amazonaws.com/dnld.lime-technology.com/test/unRAIDServer.plg

charlesmunger commented 2 years ago

As a last addendum, the current recommendation for hardware acceleration with qsv does not work - "Unlike most other values, this option does not enable accelerated decoding".

Even when I had it set up with proper filters/hwdownload, I didn't see a CPU reduction, and I was unable to get rid of approx. 2.5 seconds of latency, even with lots of tuning.

NickM-27 commented 1 year ago

This is done automatically in 0.12 with the ffmpeg presets