Detection FPS stuck at 0

[melluh]

Describe the bug The detection_fps, process (frame_info) and detection_start (detectors, coral) from /debug/stats are all stuck at 0. From the FFprobes I ran (see below), it looks like one of the cameras is doing something weird. Could this be the issue?

Additionally, I cannot get hardware acceleration to work. If it is enabled, I get ENOMEM errors. I think this means the Raspberry Pi is running out of video memory, would switching to a Raspberry Pi 4 fix this?

Version of frigate 1.3 (latest available from HomeAssistant add-on store)

Config file

web_port: 5000
detectors:
  coral:
    type: edgetpu
    device: 'usb:0'
save_clips:
  clips_dir: /media/frigate
  max_seconds: 100
mqtt:
  host: core-mosquitto.local.hass.io
ffmpeg: {}
cameras:
  veranda:
    width: 1280
    height: 720
    fps: 5
    ffmpeg:
      input: 'rtsp://xxxx:xxxx@192.168.178.xx/11'
    save_clips:
      enabled: true
      pre_capture: 15
  driveway:
    width: 1920
    height: 1080
    fps: 5
    ffmpeg:
      input: 'rtsp://xxxx:xxxx@192.168.178.xx/11'
    save_clips:
      enabled: true
      pre_capture: 15
objects:
  track:
    - person

Logs

Fontconfig error: Cannot load default config file
On connect called
Starting detection process: 16
Attempting to load TPU as usb:0
Camera capture process started for veranda: 18
Camera process started for veranda: 19
Camera capture process started for driveway: 22
Creating ffmpeg process...
ffmpeg -hide_banner -loglevel panic -avoid_negative_ts make_zero -fflags nobuffer -flags low_delay -strict experimental -fflags +genpts+discardcorrupt -rtsp_transport tcp -stimeout 5000000 -use_wallclock_as_timestamps 1 -i rtsp://****:****@192.168.178.**/11 -f segment -segment_time 10 -segment_format mp4 -reset_timestamps 1 -strftime 1 -c copy -an -map 0 /cache/veranda-%Y%m%d%H%M%S.mp4 -r 5 -f rawvideo -pix_fmt yuv420p pipe:
Camera process started for driveway: 23
Creating ffmpeg process...
ffmpeg -hide_banner -loglevel panic -avoid_negative_ts make_zero -fflags nobuffer -flags low_delay -strict experimental -fflags +genpts+discardcorrupt -rtsp_transport tcp -stimeout 5000000 -use_wallclock_as_timestamps 1 -i rtsp://****:****@192.168.178.**/11 -f segment -segment_time 10 -segment_format mp4 -reset_timestamps 1 -strftime 1 -c copy -an -map 0 /cache/driveway-%Y%m%d%H%M%S.mp4 -r 5 -f rawvideo -pix_fmt yuv420p pipe:
 * Serving Flask app "detect_objects" (lazy loading)
 * Environment: development
 * Debug mode: off
TPU found

Frigate debug stats

{"detection_fps":0.0,"detectors":{"coral":{"detection_start":0.0,"inference_speed":61.47,"pid":16}},"driveway":{"camera_fps":5.0,"capture_pid":22,"detection_fps":0.0,"frame_info":{"detect":1606403051.453198,"process":0.0},"pid":23,"process_fps":5.0,"skipped_fps":0.0},"veranda":{"camera_fps":5.1,"capture_pid":18,"detection_fps":0.0,"frame_info":{"detect":1606403051.592759,"process":0.0},"pid":19,"process_fps":5.1,"skipped_fps":0.0}}

FFprobe from your camera (I ran both of these from Windows, but I don't think that should make a difference)

veranda:

C:\Users\Melle>ffprobe rtsp://xxxx:xxxx@192.168.178.xx/11
ffprobe version N-86977-g5859b5b Copyright (c) 2007-2017 the FFmpeg developers
  built with gcc 7.1.0 (GCC)
  configuration: --enable-gpl --enable-version3 --enable-cuda --enable-cuvid --enable-d3d11va --enable-dxva2 --enable-libmfx --enable-nvenc --enable-avisynth --enable-bzlib --enable-fontconfig --enable-frei0r --enable-gnutls --enable-iconv --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libfreetype --enable-libgme --enable-libgsm --enable-libilbc --enable-libmodplug --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenh264 --enable-libopenjpeg --enable-libopus --enable-librtmp --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvo-amrwbenc --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxavs --enable-libxvid --enable-libzimg --enable-lzma --enable-zlib
  libavutil      55. 73.100 / 55. 73.100
  libavcodec     57.102.100 / 57.102.100
  libavformat    57. 76.100 / 57. 76.100
  libavdevice    57.  7.100 / 57.  7.100
  libavfilter     6. 98.100 /  6. 98.100
  libswscale      4.  7.102 /  4.  7.102
  libswresample   2.  8.100 /  2.  8.100
  libpostproc    54.  6.100 / 54.  6.100
[udp @ 00000000001eb7c0] 'circular_buffer_size' option was set but it is not supported on this build (pthread support is required)
[udp @ 00000000001ee300] 'circular_buffer_size' option was set but it is not supported on this build (pthread support is required)
[udp @ 00000000001eef00] 'circular_buffer_size' option was set but it is not supported on this build (pthread support is required)
[udp @ 0000000002634d00] 'circular_buffer_size' option was set but it is not supported on this build (pthread support is required)
[h264 @ 00000000001ec760] illegal reordering_of_pic_nums_idc 9
[h264 @ 00000000001ec760] illegal modification_of_pic_nums_idc 9
[h264 @ 00000000001ec760] decode_slice_header error
[h264 @ 00000000001ec760] no frame!
[h264 @ 00000000001ec760] reference overflow 20 > 15 or 0 > 15
    Last message repeated 1 times
[h264 @ 00000000001ec760] decode_slice_header error
[h264 @ 00000000001ec760] no frame!
[h264 @ 00000000001ec760] top block unavailable for requested intra mode
[h264 @ 00000000001ec760] error while decoding MB 62 0, bytestream 7057
[h264 @ 00000000001ec760] concealing 3587 DC, 3587 AC, 3587 MV errors in P frame
[h264 @ 00000000001ec760] cabac_init_idc 4 overflow
[h264 @ 00000000001ec760] decode_slice_header error
[h264 @ 00000000001ec760] no frame!
[h264 @ 00000000001ec760] illegal reordering_of_pic_nums_idc 9
[h264 @ 00000000001ec760] illegal modification_of_pic_nums_idc 9
[h264 @ 00000000001ec760] decode_slice_header error
[h264 @ 00000000001ec760] no frame!
[h264 @ 00000000001ec760] reference overflow 20 > 15 or 0 > 15
    Last message repeated 1 times
[h264 @ 00000000001ec760] decode_slice_header error
[h264 @ 00000000001ec760] no frame!
[h264 @ 00000000001ec760] top block unavailable for requested intra mode -1
[h264 @ 00000000001ec760] error while decoding MB 6 0, bytestream 7558
[h264 @ 00000000001ec760] concealing 3600 DC, 3600 AC, 3600 MV errors in P frame
[h264 @ 00000000001ec760] cabac_init_idc 4 overflow
[h264 @ 00000000001ec760] decode_slice_header error
[h264 @ 00000000001ec760] no frame!
[h264 @ 00000000001ec760] illegal reordering_of_pic_nums_idc 9
[h264 @ 00000000001ec760] illegal modification_of_pic_nums_idc 9
[h264 @ 00000000001ec760] decode_slice_header error
[h264 @ 00000000001ec760] no frame!
[h264 @ 00000000001ec760] reference overflow 20 > 15 or 0 > 15
    Last message repeated 1 times
[h264 @ 00000000001ec760] decode_slice_header error
[h264 @ 00000000001ec760] no frame!
[h264 @ 00000000001ec760] top block unavailable for requested intra mode
[h264 @ 00000000001ec760] error while decoding MB 42 0, bytestream 6963
[h264 @ 00000000001ec760] concealing 3600 DC, 3600 AC, 3600 MV errors in P frame
[h264 @ 00000000001ec760] cabac_init_idc 4 overflow
[h264 @ 00000000001ec760] decode_slice_header error
[h264 @ 00000000001ec760] no frame!
[h264 @ 00000000001ec760] illegal reordering_of_pic_nums_idc 9
[h264 @ 00000000001ec760] illegal modification_of_pic_nums_idc 9
[h264 @ 00000000001ec760] decode_slice_header error
[h264 @ 00000000001ec760] no frame!
[h264 @ 00000000001ec760] reference overflow 20 > 15 or 0 > 15
    Last message repeated 1 times
[h264 @ 00000000001ec760] decode_slice_header error
[h264 @ 00000000001ec760] no frame!
[h264 @ 00000000001ec760] concealing 3175 DC, 3175 AC, 3175 MV errors in P frame
[h264 @ 00000000001ec760] cabac_init_idc 4 overflow
[h264 @ 00000000001ec760] decode_slice_header error
[h264 @ 00000000001ec760] no frame!
[h264 @ 00000000001ec760] illegal reordering_of_pic_nums_idc 9
[h264 @ 00000000001ec760] illegal modification_of_pic_nums_idc 9
[h264 @ 00000000001ec760] decode_slice_header error
[h264 @ 00000000001ec760] no frame!
[h264 @ 00000000001ec760] reference overflow 30 > 15 or 0 > 15
    Last message repeated 1 times
[h264 @ 00000000001ec760] decode_slice_header error
[h264 @ 00000000001ec760] no frame!
[h264 @ 00000000001ec760] concealing 3581 DC, 3581 AC, 3581 MV errors in P frame
[h264 @ 00000000001ec760] cabac_init_idc 6 overflow
[h264 @ 00000000001ec760] decode_slice_header error
[h264 @ 00000000001ec760] no frame!
[h264 @ 00000000001ec760] illegal reordering_of_pic_nums_idc 14
[h264 @ 00000000001ec760] illegal modification_of_pic_nums_idc 14
[h264 @ 00000000001ec760] decode_slice_header error
[h264 @ 00000000001ec760] no frame!
Input #0, rtsp, from 'rtsp://xxxx:xxxx@192.168.178.xx/11':
  Metadata:
    title           : 10
  Duration: N/A, start: 0.040000, bitrate: N/A
    Stream #0:0: Video: h264 (Main), yuvj420p(pc, bt709, progressive), 1280x720, 25 tbr, 90k tbn, 180k tbc
    Stream #0:1: Audio: pcm_alaw, 8000 Hz, 1 channels, s16, 64 kb/s

driveway:

C:\Users\Melle>ffprobe rtsp://xxxx:xxxx@192.168.178.xx/11'
ffprobe version N-86977-g5859b5b Copyright (c) 2007-2017 the FFmpeg developers
  built with gcc 7.1.0 (GCC)
  configuration: --enable-gpl --enable-version3 --enable-cuda --enable-cuvid --enable-d3d11va --enable-dxva2 --enable-libmfx --enable-nvenc --enable-avisynth --enable-bzlib --enable-fontconfig --enable-frei0r --enable-gnutls --enable-iconv --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libfreetype --enable-libgme --enable-libgsm --enable-libilbc --enable-libmodplug --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenh264 --enable-libopenjpeg --enable-libopus --enable-librtmp --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvo-amrwbenc --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxavs --enable-libxvid --enable-libzimg --enable-lzma --enable-zlib
  libavutil      55. 73.100 / 55. 73.100
  libavcodec     57.102.100 / 57.102.100
  libavformat    57. 76.100 / 57. 76.100
  libavdevice    57.  7.100 / 57.  7.100
  libavfilter     6. 98.100 /  6. 98.100
  libswscale      4.  7.102 /  4.  7.102
  libswresample   2.  8.100 /  2.  8.100
  libpostproc    54.  6.100 / 54.  6.100
[udp @ 0000000000dab7c0] 'circular_buffer_size' option was set but it is not supported on this build (pthread support is required)
[udp @ 0000000000dae300] 'circular_buffer_size' option was set but it is not supported on this build (pthread support is required)
[udp @ 0000000000daef00] 'circular_buffer_size' option was set but it is not supported on this build (pthread support is required)
[udp @ 0000000002684d00] 'circular_buffer_size' option was set but it is not supported on this build (pthread support is required)
[rtsp @ 0000000002653560] max delay reached. need to consume packet
[rtsp @ 0000000002653560] RTP: missed 21 packets
Input #0, rtsp, from 'rtsp://xxxx:xxxx@192.168.178.xx/11':
  Metadata:
    title           : 10
  Duration: N/A, start: 0.240000, bitrate: N/A
    Stream #0:0: Video: h264 (Main), yuvj420p(pc, bt709, progressive), 1920x1080, 12.50 tbr, 90k tbn, 180k tbc
    Stream #0:1: Audio: pcm_alaw, 8000 Hz, 1 channels, s16, 64 kb/s

Computer Hardware

• OS: HassIO
• Virtualization: HomeAssistant (protection mode disabled)
• Coral Version: USB
• Network Setup: Wired

Camera Info:

• Manufacturer: Eminent (probably just rebranded)
• Model: EM6355 (driveway), EM6360 (veranda)
• Resolution: 1080p (driveway), 720p (veranda)
• FPS: 30

[blakeblackshear]

detection_fps will be 0 unless frigate detects motion. detection_start should be 0 most of the time during normal operation. Can you see the video feed using the mjpeg endpoint?

[melluh]

Oops, seems I confused detection and process FPS. Sorry!