search

blakeblackshear / frigate-hass-addons

Frigate hass.io addons
MIT License
225 stars 122 forks source link

Trying to enable hardware acceleration on GPU (Intel 11th gen) just get green screen #82

Open Cardy165 opened 2 years ago

Cardy165 commented 2 years ago

I'm wonering if anyone has this working or if this is even possible.

I have the Hassos addon running inside the lastest version of HassOS.

I recently upgraded my hardware and HassOS runs directly on the hardware which is a 11th Gen Intel(R) Core(TM) i5-11400T.

I use a coral device for object detection which is working fine. I don't know if using the hardware acceleration is possible with the addon vs running frigate in a dedicated docker container as I have seen else where ?

I have had the camera's running for a long time and all work well but ffmpeg chews quite a bit of CPU and I wanted to enable the hardware acceleration support to reduce this.

I have checked the addon by connecting to the docker with a bash shell and I can see the /dev/dri/renderD128 device is available.

When i try to enable the hw acceleration I just get a green screen on each camera. Disable hwacceleration and it works all be it with high CPU. I don't seem to see any information about ffmpeg using the hardware for processing however.

frigate.yaml (ffmpeg)

ffmpeg:
  # Optional: global ffmpeg args (default: shown below)
  #global_args: -hide_banner -loglevel warning
  global_args: -hide_banner -loglevel verbose
  #hwaccel_args: []

# QSV Settings
  hwaccel_args:
    - -hwaccel
    - qsv
    - -hwaccel_device
    - /dev/dri/renderD128
    - -hwaccel_output_format
    - yuv420p
    - -c:v
    - h264_qsv

# VAAPI Settings
  # hwaccel_args:  
  # - -hwaccel 
  # - vaapi 
  # - -hwaccel_device 
  # - /dev/dri/renderD128 
  # - -hwaccel_output_format 
  # - vaapi

# Alternate VAAPI Settings
  # hwaccel_args:
  # - -init_hw_device 
  # - vaapi=intel:/dev/dri/renderD128
  # - -hwaccel
  # - vaapi
  # - -hwaccel_device
  # - intel
#   - -hwaccel_output_format
#   - yuv420p

  input_args: -avoid_negative_ts make_zero -fflags +genpts+discardcorrupt -rtsp_transport tcp -timeout 5000000 -use_wallclock_as_timestamps 1

  # Optional: global output args

  output_args:

    detect: -f rawvideo -pix_fmt yuv420p
    # Optional: output args for record streams (default: shown below)
    record: -f segment -segment_time 60 -segment_format mp4 -reset_timestamps 1 -strftime 1 -c copy -an
    # Optional: output args for clips streams (default: shown below)
    #clips: -f segment -segment_time 10 -segment_format mp4 -reset_timestamps 1 -strftime 1 -c copy -an
    # Optional: output args for rtmp streams (default: shown below)
    rtmp: -c copy -f flv

Startup log:

[s6-init] making user provided files available at /var/run/s6/etc...exited 0.
[s6-init] ensuring user provided files have correct perms...exited 0.
[fix-attrs.d] applying ownership & permissions fixes...
[fix-attrs.d] done.
[cont-init.d] executing container initialization scripts...
[cont-init.d] done.
[services.d] starting services
[services.d] done.
[2022-10-04 11:38:19] frigate.app                    INFO    : Starting Frigate (0.11.0-3846a13)
[2022-10-04 11:38:19] frigate.app                    INFO    : Creating directory: /tmp/cache
Starting migrations
[2022-10-04 11:38:19] peewee_migrate                 INFO    : Starting migrations
There is nothing to migrate
[2022-10-04 11:38:19] peewee_migrate                 INFO    : There is nothing to migrate
[2022-10-04 11:38:19] detector.coral                 INFO    : Starting detection process: 232
[2022-10-04 11:38:19] frigate.app                    INFO    : Output process started: 233
[2022-10-04 11:38:19] ws4py                          INFO    : Using epoll
[2022-10-04 11:38:19] frigate.edgetpu                INFO    : Attempting to load TPU as usb
[2022-10-04 11:38:19] frigate.app                    INFO    : Camera processor started for living_room: 239
[2022-10-04 11:38:22] frigate.edgetpu                INFO    : TPU found
[2022-10-04 11:38:19] frigate.app                    INFO    : Camera processor started for study: 241
[2022-10-04 11:38:19] frigate.app                    INFO    : Camera processor started for kids_room: 245
[2022-10-04 11:38:19] frigate.app                    INFO    : Camera processor started for front_door: 247
[2022-10-04 11:38:19] frigate.app                    INFO    : Camera processor started for back_garden: 249
[2022-10-04 11:38:19] frigate.app                    INFO    : Camera processor started for kitchen: 250
[2022-10-04 11:38:19] frigate.app                    INFO    : Camera processor started for parking: 252
[2022-10-04 11:38:19] frigate.app                    INFO    : Capture process started for living_room: 253
[2022-10-04 11:38:19] frigate.app                    INFO    : Capture process started for study: 255
[2022-10-04 11:38:19] frigate.app                    INFO    : Capture process started for kids_room: 259
[2022-10-04 11:38:19] frigate.app                    INFO    : Capture process started for front_door: 264
[2022-10-04 11:38:19] frigate.app                    INFO    : Capture process started for back_garden: 284
[2022-10-04 11:38:19] frigate.app                    INFO    : Capture process started for kitchen: 292
[2022-10-04 11:38:19] frigate.app                    INFO    : Capture process started for parking: 301
[2022-10-04 11:38:19] ws4py                          INFO    : Using epoll
[2022-10-04 11:38:39] watchdog.living_room           INFO    : No frames received from living_room in 20 seconds. Exiting ffmpeg...
[2022-10-04 11:38:39] watchdog.living_room           INFO    : Waiting for ffmpeg to exit gracefully...
[2022-10-04 11:38:39] watchdog.study                 INFO    : No frames received from study in 20 seconds. Exiting ffmpeg...
[2022-10-04 11:38:39] watchdog.study                 INFO    : Waiting for ffmpeg to exit gracefully...
[2022-10-04 11:38:39] watchdog.kids_room             INFO    : No frames received from kids_room in 20 seconds. Exiting ffmpeg...
[2022-10-04 11:38:39] watchdog.kids_room             INFO    : Waiting for ffmpeg to exit gracefully...
[2022-10-04 11:38:39] watchdog.front_door            INFO    : No frames received from front_door in 20 seconds. Exiting ffmpeg...
[2022-10-04 11:38:39] watchdog.front_door            INFO    : Waiting for ffmpeg to exit gracefully...
[2022-10-04 11:38:39] watchdog.kitchen               INFO    : No frames received from kitchen in 20 seconds. Exiting ffmpeg...
[2022-10-04 11:38:39] watchdog.back_garden           INFO    : No frames received from back_garden in 20 seconds. Exiting ffmpeg...
[2022-10-04 11:38:39] watchdog.kitchen               INFO    : Waiting for ffmpeg to exit gracefully...
[2022-10-04 11:38:39] watchdog.back_garden           INFO    : Waiting for ffmpeg to exit gracefully...
[2022-10-04 11:38:39] frigate.video                  ERROR   : kids_room: Unable to read frames from ffmpeg process.
[2022-10-04 11:38:39] frigate.video                  ERROR   : kids_room: ffmpeg process is not running. exiting capture thread...
[2022-10-04 11:38:39] frigate.video                  ERROR   : front_door: Unable to read frames from ffmpeg process.
[2022-10-04 11:38:39] frigate.video                  ERROR   : front_door: ffmpeg process is not running. exiting capture thread...
[2022-10-04 11:38:39] frigate.video                  ERROR   : back_garden: Unable to read frames from ffmpeg process.
[2022-10-04 11:38:39] frigate.video                  ERROR   : back_garden: ffmpeg process is not running. exiting capture thread...
[2022-10-04 11:38:39] watchdog.parking               INFO    : No frames received from parking in 20 seconds. Exiting ffmpeg...
[2022-10-04 11:38:39] watchdog.parking               INFO    : Waiting for ffmpeg to exit gracefully...
[2022-10-04 11:38:39] frigate.video                  ERROR   : kitchen: Unable to read frames from ffmpeg process.
[2022-10-04 11:38:39] frigate.video                  ERROR   : kitchen: ffmpeg process is not running. exiting capture thread...
[2022-10-04 11:38:39] frigate.video                  ERROR   : study: Unable to read frames from ffmpeg process.
[2022-10-04 11:38:39] frigate.video                  ERROR   : study: ffmpeg process is not running. exiting capture thread...
[2022-10-04 11:38:39] frigate.video                  ERROR   : living_room: Unable to read frames from ffmpeg process.
[2022-10-04 11:38:39] frigate.video                  ERROR   : living_room: ffmpeg process is not running. exiting capture thread...
[2022-10-04 11:38:39] frigate.video                  ERROR   : parking: Unable to read frames from ffmpeg process.
[2022-10-04 11:38:39] frigate.video                  ERROR   : parking: ffmpeg process is not running. exiting capture thread...

If I start ffmpeg manually just changing the cache name of the camera e.g. the frigate ffmpeg is showing as the process:

ffmpeg -hide_banner -loglevel verbose -hwaccel qsv -hwaccel_device /dev/dri/renderD128 -hwaccel_output_format yuv420p -c:v h264_qsv -avoid_negative_ts make_zero -fflags +genpts+discardcorrupt -rtsp_transport tcp -timeout 5000000 -use_wallclock_as_timestamps 1 -i rtsp://REDACTED:REDACTED@192.168.1.84:554/Streaming/Channels/101 -f segment -segment_time 60 -segment_format mp4 -reset_timestamps 1 -strftime 1 -c copy -an /tmp/cache/back_garden-%Y%m%d%H%M%S.mp4 -c copy -f flv rtmp://127.0.0.1/live/back_garden

If I use docker exec -it bash on the container for frigate and run the above command just changing the name for cache and rtmp as below

ffmpeg -hide_banner -loglevel verbose -hwaccel qsv -hwaccel_device /dev/dri/renderD128 -hwaccel_output_format yuv420p -c:v h264_qsv -avoid_negative_ts make_zero -fflags +genpts+discardcorrupt -rtsp_transport tcp -timeout 5000000 -use_wallclock_as_timestamps 1 -i rtsp://REDACTED:REDACTED@192.168.1.84:554/Streaming/Channels/101 -f segment -segment_time 60 -segment_format mp4 -reset_timestamps 1 -strftime 1 -c copy -an /tmp/cache/test1-%Y%m%d%H%M%S.mp4 -c copy -f flv rtmp://127.0.0.1/live/test1

I get the following:

root@ccab4aaf-frigate:/opt/frigate# ffmpeg -hide_banner -loglevel verbose -hwaccel qsv -hwaccel_device /dev/dri/renderD128 -hwaccel_output_format yuv420p -c:v h264_qsv -avoid_negative_ts make_zero -fflags +genpts+discardcorrupt -rtsp_transport tcp -timeout 5000000 -use_wallclock_as_timestamps 1 -i rtsp://REDACTED:REDACTED@192.168.1.84:554/Streaming/Channels/101 -f segment -segment_time 60 -segment_format mp4 -reset_timestamps 1 -strftime 1 -c copy -an /tmp/cache/test1-%Y%m%d%H%M%S.mp4 -c copy -f flv rtmp://127.0.0.1/live/test1
[tcp @ 0x55d359d5c280] Starting connection attempt to 192.168.1.84 port 554
[tcp @ 0x55d359d5c280] Successfully connected to 192.168.1.84 port 554
[rtsp @ 0x55d359d58fc0] SDP:
v=0
o=- 1664880513970817 1664880513970817 IN IP4 192.168.1.84
s=Media Presentation
e=NONE
b=AS:5100
t=0 0
a=control:rtsp://192.168.1.84:554/Streaming/Channels/101/
m=video 0 RTP/AVP 96
b=AS:5000
a=control:rtsp://192.168.1.84:554/Streaming/Channels/101/trackID=1
a=rtpmap:96 H264/90000
a=fmtp:96 profile-level-id=420029; packetization-mode=1; sprop-parameter-sets=Z00AKY2NQDwBE/LCAAARlAACvyAI,aO44gA==
m=audio 0 RTP/AVP 104
b=AS:50
a=control:rtsp://192.168.1.84:554/Streaming/Channels/101/trackID=2
a=rtpmap:104 mpeg4-generic/16000/1
a=fmtp:104 profile-level-id=15; streamtype=5; mode=AAC-hbr; config=1408;SizeLength=13; IndexLength=3; IndexDeltaLength=3; Profile=1;
a=Media_header:MEDIAINFO=494D4B48010100000400010001200110803E0000007D000000000000000000000000000000000000;
a=appversion:1.0

[rtsp @ 0x55d359d58fc0] setting jitter buffer size to 0
    Last message repeated 1 times
[h264 @ 0x55d359d5e400] Reinit context to 1920x1088, pix_fmt: yuv420p
Input #0, rtsp, from 'rtsp://REDACTED:REDACTED@192.168.1.84:554/Streaming/Channels/101':
  Metadata:
    title           : Media Presentation
  Duration: N/A, start: 1664880515.383563, bitrate: N/A
  Stream #0:0: Video: h264 (Main), 1 reference frame, yuv420p(progressive, left), 1920x1080 (1920x1088), 20 fps, 20 tbr, 90k tbn
  Stream #0:1: Audio: aac (LC), 16000 Hz, mono, fltp
[tcp @ 0x55d359d78940] Starting connection attempt to 127.0.0.1 port 1935
[tcp @ 0x55d359d78940] Successfully connected to 127.0.0.1 port 1935
[segment @ 0x55d359d75080] Selected stream id:0 type:video
[segment @ 0x55d359d75080] Opening '/tmp/cache/test1-20221004114836.mp4' for writing
Output #0, segment, to '/tmp/cache/test1-%Y%m%d%H%M%S.mp4':
  Metadata:
    title           : Media Presentation
    encoder         : Lavf59.27.100
  Stream #0:0: Video: h264 (Main), 1 reference frame, yuv420p(progressive, left), 1920x1080 (0x0), q=2-31, 20 fps, 20 tbr, 10240 tbn
Output #1, flv, to 'rtmp://127.0.0.1/live/test1':
  Metadata:
    title           : Media Presentation
    encoder         : Lavf59.27.100
  Stream #1:0: Video: h264 (Main), 1 reference frame ([7][0][0][0] / 0x0007), yuv420p(progressive, left), 1920x1080 (0x0), q=2-31, 20 fps, 20 tbr, 1k tbn
  Stream #1:1: Audio: aac (LC) ([10][0][0][0] / 0x000A), 16000 Hz, mono, fltp
Stream mapping:
  Stream #0:0 -> #0:0 (copy)
  Stream #0:0 -> #1:0 (copy)
  Stream #0:1 -> #1:1 (copy)
Press [q] to stop, [?] for help
[segment @ 0x55d359d75080] Timestamps are unset in a packet for stream 0. This is deprecated and will stop working in the future. Fix your code to set the timestamps properly
[segment @ 0x55d359d75080] segment:'/tmp/cache/test1-20221004114836.mp4' starts with packet stream:0 pts:0 pts_time:0 frame:0
[flv @ 0x55d359d7a800] Timestamps are unset in a packet for stream 0. This is deprecated and will stop working in the future. Fix your code to set the timestamps properly
[segment @ 0x55d359d75080] segment:'/tmp/cache/test1-20221004114836.mp4' count:0 ended 
[AVIOContext @ 0x55d359d78a40] Statistics: 1967357 bytes written, 2 seeks, 11 writeouts
[flv @ 0x55d359d7a800] Failed to update header with correct duration.
[flv @ 0x55d359d7a800] Failed to update header with correct filesize.
frame=  302 fps= 22 q=-1.0 Lq=-1.0 size=N/A time=00:00:14.91 bitrate=N/A speed=1.07x    
video:3834kB audio:57kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown
Input file #0 (rtsp://REDACTED:REDACTED@192.168.1.84:554/Streaming/Channels/101):
  Input stream #0:0 (video): 302 packets read (1962826 bytes); 
  Input stream #0:1 (audio): 237 packets read (58019 bytes); 
  Total: 539 packets (2020845 bytes) demuxed
Output file #0 (/tmp/cache/test1-%Y%m%d%H%M%S.mp4):
  Output stream #0:0 (video): 302 packets muxed (1962826 bytes); 
  Total: 302 packets (1962826 bytes) muxed
Output file #1 (rtmp://127.0.0.1/live/test1):
  Output stream #1:0 (video): 302 packets muxed (1962826 bytes); 
  Output stream #1:1 (audio): 237 packets muxed (58019 bytes); 
  Total: 539 packets (2020845 bytes) muxed
[AVIOContext @ 0x55d359dac5c0] Statistics: 2031358 bytes written, 0 seeks, 557 writeouts
Exiting normally, received signal 2.

The exit was because I pressed CTRL+C after a short while.

As far as I can see this is only using software decode rather than hardware decode as displayed here: Stream #0:0: Video: h264 (Main), 1 reference frame, yuv420p(progressive, left), 1920x1080 (1920x1088), 20 fps, 20 tbr, 90k tbn

Any help would be fantastic as I have looked at so many options both using QSV and VAAPI but neither seem to work within the addon and I am wondering if this is even possible within the addon ?

Render device:

root@ccab4aaf-frigate:/opt/frigate# ls -l /dev/dri/           
total 0
drwxr-xr-x 2 root root       80 Oct  4 10:02 by-path
crw-rw---- 1 root   28 226,   0 Oct  4 10:02 card0
crw-rw-rw- 1 root  103 226, 128 Oct  4 10:02 renderD128

If there is more that would help please let me know I would love to get this working.

Lee

NickM-27 commented 2 years ago

What CPU do you have?

Also those QSV args are deprecated, new args are suggested https://docs.frigate.video/configuration/hardware_acceleration#intel-based-cpus-10th-generation-via-quicksync-1

Cardy165 commented 2 years ago

Hi Nick,

Thanks for coming back, I did include the CPU model in the top section but its easily missed. The CPU is: 11th Gen Intel(R) Core(TM) i5-11400T.

NickM-27 commented 2 years ago

Okay cool, definitely try the args I linked instead.

Cardy165 commented 2 years ago

Hi Nick,

I ran the updated command with the hardware options recommended within the frigate container:

ffmpeg -hide_banner -loglevel verbose -c:v h264_qsv -avoid_negative_ts make_zero -fflags +genpts+discardcorrupt -rtsp_transport tcp -timeout 5000000 -use_wallclock_as_timestamps 1 -i rtsp://REDACTED:REDACTED@192.168.222.84:554/Streaming/Channels/101 -f segment -segment_time 60 -segment_format mp4 -reset_timestamps 1 -strftime 1 -c copy -an /tmp/cache/test1-%Y%m%d%H%M%S.mp4 -c copy -f flv rtmp://127.0.0.1/live/test1

Output below:

root@ccab4aaf-frigate:/opt/frigate# ffmpeg -hide_banner -loglevel verbose -c:v h264_qsv -avoid_negative_ts make_zero -fflags +genpts+discardcorrupt -rtsp_transport tcp -timeout 5000000 -use_wallclock_as_timestamps 1 -i rtsp://REDACTED:REDACTED@192.168.1.84:554/Streaming/Channels/101 -f segment -segment_time 60 -segment_format mp4 -reset_timestamps 1 -strftime 1 -c copy -an /tmp/cache/test1-%Y%m%d%H%M%S.mp4 -c copy -f flv rtmp://127.0.0.1/live/test1
[tcp @ 0x55be2e70b180] Starting connection attempt to 192.168.1.84 port 554
[tcp @ 0x55be2e70b180] Successfully connected to 192.168.1.84 port 554
[rtsp @ 0x55be2e707ec0] SDP:
v=0
o=- 1664889138402813 1664889138402813 IN IP4 192.168.1.84
s=Media Presentation
e=NONE
b=AS:5100
t=0 0
a=control:rtsp://192.168.1.84:554/Streaming/Channels/101/
m=video 0 RTP/AVP 96
b=AS:5000
a=control:rtsp://192.168.1.84:554/Streaming/Channels/101/trackID=1
a=rtpmap:96 H264/90000
a=fmtp:96 profile-level-id=420029; packetization-mode=1; sprop-parameter-sets=Z00AKY2NQDwBE/LCAAARlAACvyAI,aO44gA==
m=audio 0 RTP/AVP 104
b=AS:50
a=control:rtsp://192.168.1.84:554/Streaming/Channels/101/trackID=2
a=rtpmap:104 mpeg4-generic/16000/1
a=fmtp:104 profile-level-id=15; streamtype=5; mode=AAC-hbr; config=1408;SizeLength=13; IndexLength=3; IndexDeltaLength=3; Profile=1;
a=Media_header:MEDIAINFO=494D4B48010100000400010001200110803E0000007D000000000000000000000000000000000000;
a=appversion:1.0

[rtsp @ 0x55be2e707ec0] setting jitter buffer size to 0
    Last message repeated 1 times
[h264 @ 0x55be2e70d300] Reinit context to 1920x1088, pix_fmt: yuv420p
Input #0, rtsp, from 'rtsp://REDACTED:REDACTED@192.168.1.84:554/Streaming/Channels/101':
  Metadata:
    title           : Media Presentation
  Duration: N/A, start: 1664889139.787375, bitrate: N/A
  Stream #0:0: Video: h264 (Main), 1 reference frame, yuv420p(progressive, left), 1920x1080 (1920x1088), 20 fps, 20 tbr, 90k tbn
  Stream #0:1: Audio: aac (LC), 16000 Hz, mono, fltp
[tcp @ 0x55be2e716380] Starting connection attempt to 127.0.0.1 port 1935
[tcp @ 0x55be2e716380] Successfully connected to 127.0.0.1 port 1935
[segment @ 0x55be2e725e80] Selected stream id:0 type:video
[segment @ 0x55be2e725e80] Opening '/tmp/cache/test1-20221004141220.mp4' for writing
Output #0, segment, to '/tmp/cache/test1-%Y%m%d%H%M%S.mp4':
  Metadata:
    title           : Media Presentation
    encoder         : Lavf59.27.100
  Stream #0:0: Video: h264 (Main), 1 reference frame, yuv420p(progressive, left), 1920x1080 (0x0), q=2-31, 20 fps, 20 tbr, 10240 tbn
Output #1, flv, to 'rtmp://127.0.0.1/live/test1':
  Metadata:
    title           : Media Presentation
    encoder         : Lavf59.27.100
  Stream #1:0: Video: h264 (Main), 1 reference frame ([7][0][0][0] / 0x0007), yuv420p(progressive, left), 1920x1080 (0x0), q=2-31, 20 fps, 20 tbr, 1k tbn
  Stream #1:1: Audio: aac (LC) ([10][0][0][0] / 0x000A), 16000 Hz, mono, fltp
Stream mapping:
  Stream #0:0 -> #0:0 (copy)
  Stream #0:0 -> #1:0 (copy)
  Stream #0:1 -> #1:1 (copy)
Press [q] to stop, [?] for help
[segment @ 0x55be2e725e80] Timestamps are unset in a packet for stream 0. This is deprecated and will stop working in the future. Fix your code to set the timestamps properly
[segment @ 0x55be2e725e80] segment:'/tmp/cache/test1-20221004141220.mp4' starts with packet stream:0 pts:0 pts_time:0 frame:0
[flv @ 0x55be2e723980] Timestamps are unset in a packet for stream 0. This is deprecated and will stop working in the future. Fix your code to set the timestamps properly
[segment @ 0x55be2e725e80] Non-monotonous DTS in output stream 0:0; previous: 3034, current: 3034; changing to 3035. This may result in incorrect timestamps in the output file.
[segment @ 0x55be2e725e80] Non-monotonous DTS in output stream 0:0; previous: 11020, current: 11020; changing to 11021. This may result in incorrect timestamps in the output file.
[segment @ 0x55be2e725e80] segment:'/tmp/cache/test1-20221004141220.mp4' count:0 ended 
[AVIOContext @ 0x55be2e75a900] Statistics: 1343300 bytes written, 2 seeks, 9 writeouts
[flv @ 0x55be2e723980] Failed to update header with correct duration.
[flv @ 0x55be2e723980] Failed to update header with correct filesize.
frame=  202 fps= 23 q=-1.0 Lq=-1.0 size=N/A time=00:00:10.03 bitrate=N/A speed=1.13x    
video:2617kB audio:38kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown
Input file #0 (rtsp://REDACTED:REDACTED@192.168.1.84:554/Streaming/Channels/101):
  Input stream #0:0 (video): 202 packets read (1339989 bytes); 
  Input stream #0:1 (audio): 159 packets read (39036 bytes); 
  Total: 361 packets (1379025 bytes) demuxed
Output file #0 (/tmp/cache/test1-%Y%m%d%H%M%S.mp4):
  Output stream #0:0 (video): 202 packets muxed (1339989 bytes); 
  Total: 202 packets (1339989 bytes) muxed
Output file #1 (rtmp://127.0.0.1/live/test1):
  Output stream #1:0 (video): 202 packets muxed (1339989 bytes); 
  Output stream #1:1 (audio): 159 packets muxed (39036 bytes); 
  Total: 361 packets (1379025 bytes) muxed
[AVIOContext @ 0x55be2e7295c0] Statistics: 1386212 bytes written, 0 seeks, 371 writeouts
Exiting normally, received signal 2.

The Frigate log just shows:

[2022-10-04 14:21:41] frigate.app                    INFO    : Starting Frigate (0.11.0-3846a13)
[2022-10-04 14:21:41] frigate.app                    INFO    : Creating directory: /tmp/cache
Starting migrations
[2022-10-04 14:21:41] peewee_migrate                 INFO    : Starting migrations
There is nothing to migrate
[2022-10-04 14:21:41] peewee_migrate                 INFO    : There is nothing to migrate
[2022-10-04 14:21:41] detector.coral                 INFO    : Starting detection process: 232
[2022-10-04 14:21:41] frigate.app                    INFO    : Output process started: 234
[2022-10-04 14:21:41] frigate.edgetpu                INFO    : Attempting to load TPU as usb
[2022-10-04 14:21:41] ws4py                          INFO    : Using epoll
[2022-10-04 14:21:41] frigate.app                    INFO    : Camera processor started for living_room: 239
[2022-10-04 14:21:41] frigate.app                    INFO    : Camera processor started for study: 242
[2022-10-04 14:21:41] frigate.app                    INFO    : Camera processor started for kids_room: 244
[2022-10-04 14:21:41] frigate.app                    INFO    : Camera processor started for front_door: 246
[2022-10-04 14:21:41] frigate.app                    INFO    : Camera processor started for back_garden: 248
[2022-10-04 14:21:41] frigate.app                    INFO    : Camera processor started for kitchen: 249
[2022-10-04 14:21:41] frigate.app                    INFO    : Camera processor started for parking: 251
[2022-10-04 14:21:41] frigate.app                    INFO    : Capture process started for living_room: 252
[2022-10-04 14:21:41] frigate.app                    INFO    : Capture process started for study: 255
[2022-10-04 14:21:41] frigate.app                    INFO    : Capture process started for kids_room: 258
[2022-10-04 14:21:41] frigate.app                    INFO    : Capture process started for front_door: 263
[2022-10-04 14:21:41] frigate.app                    INFO    : Capture process started for back_garden: 273
[2022-10-04 14:21:41] frigate.app                    INFO    : Capture process started for kitchen: 285
[2022-10-04 14:21:41] frigate.app                    INFO    : Capture process started for parking: 292
[2022-10-04 14:21:42] ws4py                          INFO    : Using epoll
[2022-10-04 14:21:44] frigate.edgetpu                INFO    : TPU found
[2022-10-04 14:22:01] watchdog.living_room           INFO    : No frames received from living_room in 20 seconds. Exiting ffmpeg...
[2022-10-04 14:22:01] watchdog.living_room           INFO    : Waiting for ffmpeg to exit gracefully...
[2022-10-04 14:22:01] watchdog.study                 INFO    : No frames received from study in 20 seconds. Exiting ffmpeg...
[2022-10-04 14:22:01] watchdog.study                 INFO    : Waiting for ffmpeg to exit gracefully...
[2022-10-04 14:22:01] watchdog.kids_room             INFO    : No frames received from kids_room in 20 seconds. Exiting ffmpeg...
[2022-10-04 14:22:01] watchdog.kids_room             INFO    : Waiting for ffmpeg to exit gracefully...
[2022-10-04 14:22:01] watchdog.front_door            INFO    : No frames received from front_door in 20 seconds. Exiting ffmpeg...
[2022-10-04 14:22:01] watchdog.front_door            INFO    : Waiting for ffmpeg to exit gracefully...
[2022-10-04 14:22:01] watchdog.back_garden           INFO    : No frames received from back_garden in 20 seconds. Exiting ffmpeg...
[2022-10-04 14:22:01] watchdog.back_garden           INFO    : Waiting for ffmpeg to exit gracefully...
[2022-10-04 14:22:01] watchdog.kitchen               INFO    : No frames received from kitchen in 20 seconds. Exiting ffmpeg...
NickM-27 commented 2 years ago

Try running vainfo in the container and paste the output here

Cardy165 commented 2 years ago

I had tried that but vainfo does not exist in the container.

# docker exec -it 495914c8e5a6 bash -l
root@ccab4aaf-frigate:/opt/frigate# vainfo
bash: vainfo: command not found
root@ccab4aaf-frigate:/opt/frigate# find / -iname vainfo
NickM-27 commented 2 years ago

okay then run apt-get update apt-get install vainfo -y vainfo

Cardy165 commented 2 years ago

I just tried installing it into the container and the output is below:

root@ccab4aaf-frigate:/opt/frigate# apt-get update
Get:1 http://deb.debian.org/debian bullseye InRelease [116 kB]
Get:2 http://raspbian.raspberrypi.org/raspbian bullseye InRelease [15.0 kB]                                       
Get:3 http://deb.debian.org/debian-security bullseye-security InRelease [48.4 kB]
Get:4 http://deb.debian.org/debian bullseye-updates InRelease [44.1 kB]
Get:5 http://deb.debian.org/debian bullseye/contrib amd64 Packages [50.6 kB]
Get:6 http://deb.debian.org/debian bullseye/non-free amd64 Packages [97.7 kB]
Get:7 http://deb.debian.org/debian bullseye/main amd64 Packages [8184 kB]
Get:8 https://packages.cloud.google.com/apt coral-edgetpu-stable InRelease [6722 B]
Ign:9 https://packages.cloud.google.com/apt coral-edgetpu-stable/main amd64 Packages
Get:10 http://deb.debian.org/debian-security bullseye-security/main amd64 Packages [186 kB]
Get:11 http://deb.debian.org/debian-security bullseye-security/non-free amd64 Packages [528 B]
Get:12 http://deb.debian.org/debian bullseye-updates/main amd64 Packages [6344 B]
Get:9 https://packages.cloud.google.com/apt coral-edgetpu-stable/main amd64 Packages [2317 B]
Fetched 8758 kB in 2s (4728 kB/s)                       
Reading package lists... Done
N: Skipping acquire of configured file 'main/binary-amd64/Packages' as repository 'http://raspbian.raspberrypi.org/raspbian bullseye InRelease' doesn't support architecture 'amd64'
N: Skipping acquire of configured file 'non-free/binary-amd64/Packages' as repository 'http://raspbian.raspberrypi.org/raspbian bullseye InRelease' doesn't support architecture 'amd64'
N: Skipping acquire of configured file 'contrib/binary-amd64/Packages' as repository 'http://raspbian.raspberrypi.org/raspbian bullseye InRelease' doesn't support architecture 'amd64'
N: Skipping acquire of configured file 'rpi/binary-amd64/Packages' as repository 'http://raspbian.raspberrypi.org/raspbian bullseye InRelease' doesn't support architecture 'amd64'
root@ccab4aaf-frigate:/opt/frigate# apt search vainfo
Sorting... Done
Full Text Search... Done
vainfo/stable 2.10.0+ds1-1 amd64
  Video Acceleration (VA) API for Linux -- info program```
Installed into container

```root@ccab4aaf-frigate:/opt/frigate# apt install vainfo
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
The following additional packages will be installed:
  libva-wayland2 libva-x11-2 libwayland-client0 libxext6 libxfixes3
The following NEW packages will be installed:
  libva-wayland2 libva-x11-2 libwayland-client0 libxext6 libxfixes3 vainfo
0 upgraded, 6 newly installed, 0 to remove and 2 not upgraded.
Need to get 166 kB of archives.
After this operation, 446 kB of additional disk space will be used.
Do you want to continue? [Y/n] y
Get:1 http://deb.debian.org/debian bullseye/main amd64 libwayland-client0 amd64 1.18.0-2~exp1.1 [26.9 kB]
Get:2 http://deb.debian.org/debian bullseye/main amd64 libva-wayland2 amd64 2.10.0-1 [20.9 kB]
Get:3 http://deb.debian.org/debian bullseye/main amd64 libxext6 amd64 2:1.3.3-1.1 [52.7 kB]
Get:4 http://deb.debian.org/debian bullseye/main amd64 libxfixes3 amd64 1:5.0.3-2 [22.1 kB]
Get:5 http://deb.debian.org/debian bullseye/main amd64 libva-x11-2 amd64 2.10.0-1 [24.0 kB]
Get:6 http://deb.debian.org/debian bullseye/main amd64 vainfo amd64 2.10.0+ds1-1 [19.0 kB]
Fetched 166 kB in 0s (1786 kB/s)
debconf: delaying package configuration, since apt-utils is not installed
Selecting previously unselected package libwayland-client0:amd64.
(Reading database ... 11766 files and directories currently installed.)
Preparing to unpack .../0-libwayland-client0_1.18.0-2~exp1.1_amd64.deb ...
Unpacking libwayland-client0:amd64 (1.18.0-2~exp1.1) ...
Selecting previously unselected package libva-wayland2:amd64.
Preparing to unpack .../1-libva-wayland2_2.10.0-1_amd64.deb ...
Unpacking libva-wayland2:amd64 (2.10.0-1) ...
Selecting previously unselected package libxext6:amd64.
Preparing to unpack .../2-libxext6_2%3a1.3.3-1.1_amd64.deb ...
Unpacking libxext6:amd64 (2:1.3.3-1.1) ...
Selecting previously unselected package libxfixes3:amd64.
Preparing to unpack .../3-libxfixes3_1%3a5.0.3-2_amd64.deb ...
Unpacking libxfixes3:amd64 (1:5.0.3-2) ...
Selecting previously unselected package libva-x11-2:amd64.
Preparing to unpack .../4-libva-x11-2_2.10.0-1_amd64.deb ...
Unpacking libva-x11-2:amd64 (2.10.0-1) ...
Selecting previously unselected package vainfo.
Preparing to unpack .../5-vainfo_2.10.0+ds1-1_amd64.deb ...
Unpacking vainfo (2.10.0+ds1-1) ...
Setting up libxext6:amd64 (2:1.3.3-1.1) ...
Setting up libxfixes3:amd64 (1:5.0.3-2) ...
Setting up libva-x11-2:amd64 (2.10.0-1) ...
Setting up libwayland-client0:amd64 (1.18.0-2~exp1.1) ...
Setting up libva-wayland2:amd64 (2.10.0-1) ...
Setting up vainfo (2.10.0+ds1-1) ...
Processing triggers for libc-bin (2.31-13+deb11u4) ...```

Output from vainfo:

```root@ccab4aaf-frigate:/opt/frigate# vainfo
error: XDG_RUNTIME_DIR not set in the environment.
error: can't connect to X server!
libva info: VA-API version 1.10.0
libva info: Trying to open /usr/lib/x86_64-linux-gnu/dri/iHD_drv_video.so
libva info: Found init function __vaDriverInit_1_10
libva info: va_openDriver() returns 0
vainfo: VA-API version: 1.10 (libva 2.10.0)
vainfo: Driver version: Intel iHD driver for Intel(R) Gen Graphics - 21.1.1 ()
vainfo: Supported profile and entrypoints
      VAProfileNone                   : VAEntrypointVideoProc
      VAProfileNone                   : VAEntrypointStats
      VAProfileMPEG2Simple            : VAEntrypointVLD
      VAProfileMPEG2Simple            : VAEntrypointEncSlice
      VAProfileMPEG2Main              : VAEntrypointVLD
      VAProfileMPEG2Main              : VAEntrypointEncSlice
      VAProfileH264Main               : VAEntrypointVLD
      VAProfileH264Main               : VAEntrypointEncSlice
      VAProfileH264Main               : VAEntrypointFEI
      VAProfileH264Main               : VAEntrypointEncSliceLP
      VAProfileH264High               : VAEntrypointVLD
      VAProfileH264High               : VAEntrypointEncSlice
      VAProfileH264High               : VAEntrypointFEI
      VAProfileH264High               : VAEntrypointEncSliceLP
      VAProfileVC1Simple              : VAEntrypointVLD
      VAProfileVC1Main                : VAEntrypointVLD
      VAProfileVC1Advanced            : VAEntrypointVLD
      VAProfileJPEGBaseline           : VAEntrypointVLD
      VAProfileJPEGBaseline           : VAEntrypointEncPicture
      VAProfileH264ConstrainedBaseline: VAEntrypointVLD
      VAProfileH264ConstrainedBaseline: VAEntrypointEncSlice
      VAProfileH264ConstrainedBaseline: VAEntrypointFEI
      VAProfileH264ConstrainedBaseline: VAEntrypointEncSliceLP
      VAProfileHEVCMain               : VAEntrypointVLD
      VAProfileHEVCMain               : VAEntrypointEncSlice
      VAProfileHEVCMain               : VAEntrypointFEI
      VAProfileHEVCMain               : VAEntrypointEncSliceLP
      VAProfileHEVCMain10             : VAEntrypointVLD
      VAProfileHEVCMain10             : VAEntrypointEncSlice
      VAProfileHEVCMain10             : VAEntrypointEncSliceLP
      VAProfileVP9Profile0            : VAEntrypointVLD
      VAProfileVP9Profile1            : VAEntrypointVLD
      VAProfileVP9Profile2            : VAEntrypointVLD
      VAProfileVP9Profile3            : VAEntrypointVLD
      VAProfileHEVCMain12             : VAEntrypointVLD
      VAProfileHEVCMain12             : VAEntrypointEncSlice
      VAProfileHEVCMain422_10         : VAEntrypointVLD
      VAProfileHEVCMain422_10         : VAEntrypointEncSlice
      VAProfileHEVCMain422_12         : VAEntrypointVLD
      VAProfileHEVCMain422_12         : VAEntrypointEncSlice
      VAProfileHEVCMain444            : VAEntrypointVLD
      VAProfileHEVCMain444            : VAEntrypointEncSliceLP
      VAProfileHEVCMain444_10         : VAEntrypointVLD
      VAProfileHEVCMain444_10         : VAEntrypointEncSliceLP
      VAProfileHEVCMain444_12         : VAEntrypointVLD
      VAProfileHEVCSccMain            : VAEntrypointVLD
      VAProfileHEVCSccMain            : VAEntrypointEncSliceLP
      VAProfileHEVCSccMain10          : VAEntrypointVLD
      VAProfileHEVCSccMain10          : VAEntrypointEncSliceLP
      VAProfileHEVCSccMain444         : VAEntrypointVLD
      VAProfileHEVCSccMain444         : VAEntrypointEncSliceLP
      VAProfileAV1Profile0            : VAEntrypointVLD
      VAProfileHEVCSccMain444_10      : VAEntrypointVLD
      VAProfileHEVCSccMain444_10      : VAEntrypointEncSliceLP
NickM-27 commented 2 years ago

Interesting, looks like you're probably running into https://github.com/blakeblackshear/frigate/issues/3941

If you manually update the driver it might get it working

Cardy165 commented 2 years ago

How would I go about updating the driver ?

NickM-27 commented 2 years ago

See https://github.com/blakeblackshear/frigate/issues/3941#issuecomment-1259531976

Cardy165 commented 2 years ago

MMmm I updated the driver but have not figured out how to restart the frigate to test as restarting frigate removes the new driver and the vainfo tool.

Strangely I can't get the intel-gpu-tool to run within docker but this may not be surprising I am guessing elevated privileges are required.

Do you know how I can restart frigate without restarting the docker image ?

NickM-27 commented 2 years ago

Hm, at least with standard docker the changes remain when restarting they just go away when pulling a new image. Perhaps HA OS is removing the image on each restart.

If you apt-get install intel-gpu-tool then run `intel_gpu_top it should work. What error are you getting when you try?

Cardy165 commented 2 years ago

I suspect it is doing that which makes it more tricky.

Below is what I get for intel_gpu_top

root@ccab4aaf-frigate:/opt/frigate# intel_gpu_top   
Failed to initialize PMU! (Permission denied)
NickM-27 commented 2 years ago

Are you running full access? And do you have protected mode disabled?

Cardy165 commented 2 years ago

Mmm no not running full access will try that.

Cardy165 commented 2 years ago

Thats with full access and protection mode off.

root@ccab4aaf-frigate-fa:/opt/frigate# intel_gpu_top 
Failed to initialize PMU! (Permission denied)

Running a system trace shows:

root@ccab4aaf-frigate-fa:/opt/frigate# strace intel_gpu_top
execve("/usr/bin/intel_gpu_top", ["intel_gpu_top"], 0x7ffe6c8f0d70 /* 13 vars */) = 0
brk(NULL)                               = 0x5592a7a31000
access("/etc/ld.so.preload", R_OK)      = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/etc/ld.so.cache", O_RDONLY|O_CLOEXEC) = 3
fstat(3, {st_mode=S_IFREG|0644, st_size=14342, ...}) = 0
mmap(NULL, 14342, PROT_READ, MAP_PRIVATE, 3, 0) = 0x7f3be99c3000
close(3)                                = 0
openat(AT_FDCWD, "/lib/x86_64-linux-gnu/libc.so.6", O_RDONLY|O_CLOEXEC) = 3
read(3, "\177ELF\2\1\1\3\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0@>\2\0\0\0\0\0"..., 832) = 832
fstat(3, {st_mode=S_IFREG|0755, st_size=1905632, ...}) = 0
mmap(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f3be99c1000
mmap(NULL, 1918592, PROT_READ, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7f3be97ec000
mmap(0x7f3be980e000, 1417216, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x22000) = 0x7f3be980e000
mmap(0x7f3be9968000, 323584, PROT_READ, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x17c000) = 0x7f3be9968000
mmap(0x7f3be99b7000, 24576, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x1ca000) = 0x7f3be99b7000
mmap(0x7f3be99bd000, 13952, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x7f3be99bd000
close(3)                                = 0
arch_prctl(ARCH_SET_FS, 0x7f3be99c2580) = 0
mprotect(0x7f3be99b7000, 16384, PROT_READ) = 0
mprotect(0x5592a676f000, 4096, PROT_READ) = 0
mprotect(0x7f3be99f1000, 4096, PROT_READ) = 0
munmap(0x7f3be99c3000, 14342)           = 0
ioctl(1, TCGETS, {B38400 opost isig icanon echo ...}) = 0
brk(NULL)                               = 0x5592a7a31000
brk(0x5592a7a52000)                     = 0x5592a7a52000
openat(AT_FDCWD, "/sys/devices/i915/events", O_RDONLY|O_NONBLOCK|O_CLOEXEC|O_DIRECTORY) = 3
fstat(3, {st_mode=S_IFDIR|0755, st_size=0, ...}) = 0
getdents64(3, 0x5592a7a31500 /* 35 entries */, 32768) = 1296
openat(3, "rcs0-busy", O_RDONLY)        = 4
read(4, "config=0x0\n", 128)            = 11
close(4)                                = 0
openat(3, "vecs0-busy", O_RDONLY)       = 4
read(4, "config=0x3000\n", 128)         = 14
close(4)                                = 0
openat(3, "vcs0-busy", O_RDONLY)        = 4
read(4, "config=0x2000\n", 128)         = 14
close(4)                                = 0
openat(3, "bcs0-busy", O_RDONLY)        = 4
read(4, "config=0x1000\n", 128)         = 14
close(4)                                = 0
getdents64(3, 0x5592a7a31500 /* 0 entries */, 32768) = 0
openat(AT_FDCWD, "/sys/bus/event_source/devices/i915/type", O_RDONLY) = 4
read(4, "22\n", 63)                     = 3
close(4)                                = 0
openat(AT_FDCWD, "/sys/devices/system/cpu", O_RDONLY|O_NONBLOCK|O_CLOEXEC|O_DIRECTORY) = 4
fstat(4, {st_mode=S_IFDIR|0755, st_size=0, ...}) = 0
getdents64(4, 0x5592a7a3a630 /* 30 entries */, 32768) = 864
getdents64(4, 0x5592a7a3a630 /* 0 entries */, 32768) = 0
close(4)                                = 0
perf_event_open({type=0x16 /* PERF_TYPE_??? */, size=PERF_ATTR_SIZE_VER6, config=0x100002, ...}, -1, 0, -1, 0) = -1 EACCES (Permission denied)
write(2, "Failed to initialize PMU! (Permi"..., 46Failed to initialize PMU! (Permission denied)
) = 46
exit_group(1)                           = ?
+++ exited with 1 +++
NickM-27 commented 2 years ago

Okay thanks for trying, that is unfortunate. To confirm, hwaccel is still not working with the full access and no protected mode?

Cardy165 commented 2 years ago

Yep that's confirmed:

With the hwaccel option: hwaccel_args: -c:v h264_qsv

I get:

[2022-10-04 23:45:13] ffmpeg.kitchen.detect          ERROR   : Error while decoding stream #0:0: Input/output error
[2022-10-04 23:45:13] ffmpeg.kitchen.detect          ERROR   : cur_dts is invalid st:0 (0) [init:0 i_done:0 finish:0] (this is harmless if it occurs once at the start per stream)
[2022-10-04 23:45:13] ffmpeg.kitchen.detect          ERROR   :     Last message repeated 2 times
[2022-10-04 23:45:13] ffmpeg.kitchen.detect          ERROR   : [h264_qsv @ 0x55d44194d440] Error during QSV decoding.: device failed (-17)
[2022-10-04 23:45:13] ffmpeg.kitchen.detect          ERROR   : Error while decoding stream #0:0: Input/output error
[2022-10-04 23:45:13] ffmpeg.kitchen.detect          ERROR   : cur_dts is invalid st:0 (0) [init:0 i_done:0 finish:0] (this is harmless if it occurs once at the start per stream)
[2022-10-04 23:45:13] ffmpeg.kitchen.detect          ERROR   :     Last message repeated 2 times
Plus a number of other messages

with hwaccel args:

hwaccel_args:

I get:

[2022-10-04 23:49:28] ffmpeg.kitchen.detect          ERROR   : [h264 @ 0x55e207c707c0] hardware accelerator failed to decode picture
[2022-10-04 23:49:28] ffmpeg.kitchen.detect          ERROR   : cur_dts is invalid st:0 (0) [init:0 i_done:0 finish:0] (this is harmless if it occurs once at the start per stream)
[2022-10-04 23:49:28] ffmpeg.kitchen.detect          ERROR   : Error while decoding stream #0:0: Input/output error
[2022-10-04 23:49:28] ffmpeg.kitchen.detect          ERROR   : [h264 @ 0x55e207be8a40] nal_unit_type: 1(Coded slice of a non-IDR picture), nal_ref_idc: 2
[2022-10-04 23:49:28] ffmpeg.kitchen.detect          ERROR   : cur_dts is invalid st:0 (0) [init:0 i_done:0 finish:0] (this is harmless if it occurs once at the start per stream)
[2022-10-04 23:49:28] ffmpeg.kitchen.detect          ERROR   : [h264 @ 0x55e207be8a40] Param buffer (type 0, 672 bytes) is 0.
[2022-10-04 23:49:28] ffmpeg.kitchen.detect          ERROR   : [h264 @ 0x55e207be8a40] Param buffer (type 1, 240 bytes) is 0x1.
[2022-10-04 23:49:28] ffmpeg.kitchen.detect          ERROR   : cur_dts is invalid st:0 (0) [init:0 i_done:0 finish:0] (this is harmless if it occurs once at the start per stream)
[2022-10-04 23:49:28] ffmpeg.kitchen.detect          ERROR   : [h264 @ 0x55e207be8a40] Slice 0 param buffer (3128 bytes) is 0x2.
[2022-10-04 23:49:28] ffmpeg.kitchen.detect          ERROR   : [h264 @ 0x55e207be8a40] Slice 0 data buffer (33 bytes) is 0x3.
[2022-10-04 23:49:28] ffmpeg.kitchen.detect          ERROR   : [h264 @ 0x55e207be8a40] Decode to surface 0x16.

The above errors repeat in both cases.