search

blakeblackshear / frigate

NVR with realtime local object detection for IP cameras
https://frigate.video
MIT License
19.17k stars 1.75k forks source link

[Support]: 0.13-beta1 "No frames have been received, check error logs" #7812

Closed jonadis closed 1 year ago

jonadis commented 1 year ago

Describe the problem you are having

I'm really enjoying 0.13-beta1 however I've been seeing a lot of "no frames have been received, check error logs" on the main screen. When I click on an individual camera the feed fires up just fine, then going back to the main page sometimes that error clears some times it doesn't but there's never an issue with the camera feed, just the message on the main screen. This happens on various cameras, I'll include the FFProbe output below from one of them that's currently having issues but just about all of my cameras have done this from time to time. When looking at HomeAssistant I see the same "no frames have been received, check error logs" in the camera feed for a second or two and then it goes to the camera feed as expected. Log wise there's absolutely nothing in the log within 15+ minutes of my interaction with the web page. And that goes for the frigate logs, the go2rtc logs, and the nginx logs just have the typical "GET" messages in them.

Version

0.13.0-9185753

Frigate config file

# yaml-language-server: $schema=http://10.0.71.64:5000/api/config/schema
mqtt:
  host: 10.0.71.10
  user: DVES_USER
  password: DVES_PASS

detectors:
  #  coral:
  #  type: edgetpu
  #  device: pci:0
  # cpu1:
  #   type: cpu
  #   num_threads: 2
  # deepstack:
  #   api_url: http://10.0.71.64:32168/v1/vision/detection
  #   type: deepstack
  #   api_timeout: 1 # seconds
  # ov:
  #   type: openvino
  #   device: AUTO
  #   model:
  #     path: /openvino-model/ssdlite_mobilenet_v2.xml

  tensorrt:
    type: tensorrt
    device: 0

# model:
#   width: 300
#   height: 300
#   input_tensor: nhwc
#   input_pixel_format: bgr
#   labelmap_path: /openvino-model/coco_91cl_bkgr.txt

model:
  path: /config/model_cache/tensorrt/yolov7-320.trt
  input_tensor: nchw
  input_pixel_format: rgb
  width: 320
  height: 320

birdseye:
  mode: motion

ffmpeg:
  hwaccel_args: preset-nvidia-h265
  output_args:
    record: preset-record-generic-audio-aac

detect:
  fps: 5
  enabled: True

objects:
  track:
    - car
    - cat
    - dog
    - person  
  filters:
    car:
      min_area: 5000
      min_score: 0.6
      threshold: 0.80 
    cat:
      min_area: 1000
      min_score: 0.6
      threshold: 0.80  
    dog:
      min_area: 1000
      min_score: 0.6
      threshold: 0.80                 
    person:
      min_area: 5000
      min_score: 0.6
      threshold: 0.80    

motion:
  threshold: 25

record:
  enabled: True
  retain:
    days: 2
    mode: motion
  events:
    retain:
      default: 14
      mode: active_objects

snapshots:
  enabled: True
  timestamp: True
  bounding_box: True
  retain:
    default: 30

go2rtc:
  streams:
    chicken: rtsp://admin:admin@10.0.71.64:18554/chicken
    front_door: rtsp://admin:redacted@10.0.71.131/cam/realmonitor?channel=1&subtype=0&unicast=true&proto=Onvif
    front_door_sub: rtsp://admin:redacted@10.0.71.131/cam/realmonitor?channel=1&subtype=1&unicast=true&proto=Onvif
    front_driveway: rtsp://admin:redacted@10.0.71.155:554/Streaming/Channels/101
    front_driveway_sub: rtsp://admin:redacted@10.0.71.155:554/Streaming/Channels/102
    front_porch: rtsp://admin:redacted@10.0.71.11/cam/realmonitor?channel=1&subtype=0&unicast=true&proto=Onvif
    front_porch_sub: rtsp://admin:redacted@10.0.71.11/cam/realmonitor?channel=1&subtype=1&unicast=true&proto=Onvif
    garage: rtsp://admin:redacted@10.0.71.81:554/Streaming/Channels/101
    garage_sub: rtsp://admin:redacted@10.0.71.81:554/Streaming/Channels/102
    garage_se: http://10.0.71.199/flv?port=1935&app=bcs&stream=channel0_main.bcs&user=admin&password=redacted
    garage_se_sub: http://10.0.71.199/flv?port=1935&app=bcs&stream=channel0_ext.bcs&user=admin&password=redacted
    garage_sw: http://10.0.71.161/flv?port=1935&app=bcs&stream=channel0_main.bcs&user=admin&password=redacted
    garage_sw_sub: http://10.0.71.161/flv?port=1935&app=bcs&stream=channel0_ext.bcs&user=admin&password=redacted
#    garage_sw: rtsp://admin:redacted@10.0.71.161:554/h264Preview_01_main
#    garage_sw_sub: rtsp://admin:redacted@10.0.71.161:554/h264Preview_01_sub
    mud_room: rtsp://admin:redacted@10.0.70.176:554/Streaming/Channels/101
    mud_room_sub: rtsp://admin:redacted@10.0.70.176:554/Streaming/Channels/102
    nc400: rtsp://admin:redacted@10.0.70.234:554/Streaming/Channels/101
    nc400_sub: rtsp://admin:redacted@10.0.70.234:554/Streaming/Channels/102
    pole_barn_porch: rtsp://admin:redacted@10.0.71.100/Streaming/Channels/101
    pole_barn_porch_sub: rtsp://admin:redacted@10.0.71.100/Streaming/Channels/102
    road: rtsp://admin:redacted@10.0.71.155:554/Streaming/Channels/103

timestamp_style:
  effect: solid

cameras:
  chicken:
    birdseye:
      enabled: False
    ffmpeg:
      inputs:
        - path: rtsp://127.0.0.1:8554/chicken
          input_args: preset-rtsp-restream-low-latency
          roles:
            - record
            - detect      
  front_door:
    ffmpeg:
      inputs:
        - path: rtsp://127.0.0.1:8554/front_door
          input_args: preset-rtsp-restream-low-latency
          roles:
            - record
        - path: rtsp://127.0.0.1:8554/front_door_sub
          input_args: preset-rtsp-restream-low-latency
          roles:
            - detect            
    detect:
      width: 640
      height: 480
      fps: 30
    motion:
      mask:
        - 640,0,640,45,0,45,0,0
    zones:
      front_step:
        coordinates: 640,480,640,361,0,120,0,480
  front_driveway:
    ffmpeg:
      inputs:
        - path: rtsp://127.0.0.1:8554/front_driveway
          input_args: preset-rtsp-restream-low-latency
          roles:
            - record
        - path: rtsp://127.0.0.1:8554/front_driveway_sub
          input_args: preset-rtsp-restream-low-latency
          roles:
            - detect
    detect:
      width: 1280
      height: 720
      fps: 10
    motion:
      mask:
        - 1920,636,1920,240,952,207
    objects:
      filters:
        car:
          mask:
            - 1920,635,1920,0,467,0
        cat:
          mask:
            - 1920,635,1920,0,467,0
          max_area: 50000
        dog:
          mask:
            - 1920,635,1920,0,467,0
          max_area: 50000
        person:
          mask:
            - 1920,635,1920,0,467,0
          min_area: 10000
  front_porch:
    ffmpeg:
      inputs:
        - path: rtsp://127.0.0.1:8554/front_porch
          input_args: preset-rtsp-restream-low-latency
          roles:
            - record
        - path: rtsp://127.0.0.1:8554/front_porch_sub
          input_args: preset-rtsp-restream-low-latency
          roles:
            - detect
    detect:
      width: 640
      height: 480
      fps: 10
    zones:
      front_step:
        coordinates: 640,480,640,299,499,250,462,161,159,480
  garage:
    ffmpeg:
      inputs:
        - path: rtsp://127.0.0.1:8554/garage
          input_args: preset-rtsp-restream-low-latency
          roles:
            - record
        - path: rtsp://127.0.0.1:8554/garage_sub
          input_args: preset-rtsp-restream-low-latency
          roles:
            - detect
    detect:
      width: 640
      height: 480
      fps: 10
  garage_se:
    ffmpeg:
      inputs:
        - path: rtsp://127.0.0.1:8554/garage_se
          input_args: preset-rtsp-restream-low-latency
          roles:
            - record
        - path: rtsp://127.0.0.1:8554/garage_se_sub
          input_args: preset-rtsp-restream-low-latency
          roles:
            - detect
    detect:
      width: 896
      height: 672
      fps: 7
  garage_sw:
    ffmpeg:
      inputs:
        - path: rtsp://127.0.0.1:8554/garage_sw
          input_args: preset-rtsp-restream-low-latency
          roles:
            - record
        - path: rtsp://127.0.0.1:8554/garage_sw_sub
          input_args: preset-rtsp-restream-low-latency
          roles:
            - detect
    detect:
      width: 896
      height: 672
      fps: 7
  mud_room:
    ffmpeg:
      inputs:
        - path: rtsp://127.0.0.1:8554/mud_room
          input_args: preset-rtsp-restream-low-latency
          roles:
            - record
        - path: rtsp://127.0.0.1:8554/mud_room_sub
          input_args: preset-rtsp-restream-low-latency
          roles:
            - detect
    detect:
      width: 640
      height: 480
      fps: 10
  nc400:
    ffmpeg:
      inputs:
        - path: rtsp://127.0.0.1:8554/nc400
          input_args: preset-rtsp-restream-low-latency
          roles:
            - record
        - path: rtsp://127.0.0.1:8554/nc400_sub
          input_args: preset-rtsp-restream-low-latency
          roles:
            - detect
    detect:
      width: 640
      height: 360
      fps: 10
  pole_barn_porch:
    ffmpeg:
      inputs:
        - path: rtsp://127.0.0.1:8554/pole_barn_porch
          input_args: preset-rtsp-restream-low-latency
          roles:
            - record
        - path: rtsp://127.0.0.1:8554/pole_barn_porch_sub
          input_args: preset-rtsp-restream-low-latency
          roles:
            - detect
    detect:
      width: 1280
      height: 720
      fps: 10
    objects:
      filters:
        car:
          mask:
            - 720,720,0,720,0,0,720,0
  road:
    ffmpeg:
      inputs:
        - path: rtsp://127.0.0.1:8554/road
          input_args: preset-rtsp-restream-low-latency
          roles:
            - detect
            - record
    detect:
      width: 1280
      height: 720
      fps: 10
    objects:
      filters:
        car:
          min_score: 0.4
          threshold: 0.6
    record:
      enabled: False
    snapshots:
      required_zones:
        - optimal
    zones:
      optimal:
        coordinates: 400,375,950,375,950,150,400,150

ui:
  live_mode: mse

Relevant log output

None

FFprobe output from your camera

[{"return_code":0,"stderr":"","stdout":{"programs":[],"streams":[{"avg_frame_rate":"10/1","codec_long_name":"H.265/HEVC(HighEfficiencyVideoCoding)","display_aspect_ratio":"16:9","height":2160,"width":3840},{"avg_frame_rate":"0/0","codec_long_name":"AAC(AdvancedAudioCoding)"}]}},{"return_code":0,"stderr":"","stdout":{"programs":[],"streams":[{"avg_frame_rate":"10/1","codec_long_name":"H.265/HEVC(HighEfficiencyVideoCoding)","display_aspect_ratio":"16:9","height":720,"width":1280}]}}]

Frigate stats

{"chicken":{"camera_fps":5.1,"capture_pid":434,"detection_enabled":0,"detection_fps":0.0,"ffmpeg_pid":444,"pid":407,"process_fps":5.1,"skipped_fps":0.0},"cpu_usages":{"1":{"cmdline":"/package/admin/s6/command/s6-svscan -d4 -- /run/service","cpu":"0.0","cpu_average":"0","mem":"0.0"},"15":{"cmdline":"s6-supervise s6-linux-init-shutdownd","cpu":"0.0","cpu_average":"0","mem":"0.0"},"16":{"cmdline":"/package/admin/s6-linux-init/command/s6-linux-init-shutdownd -c /run/s6/basedir -g 3000 -C -B","cpu":"0.0","cpu_average":"0","mem":"0.0"},"24":{"cmdline":"s6-supervise frigate","cpu":"0.0","cpu_average":"0","mem":"0.0"},"25":{"cmdline":"s6-supervise nginx-log","cpu":"0.0","cpu_average":"0","mem":"0.0"},"26":{"cmdline":"s6-supervise go2rtc-log","cpu":"0.0","cpu_average":"0","mem":"0.0"},"27":{"cmdline":"s6-supervise nginx","cpu":"0.0","cpu_average":"0","mem":"0.0"},"28":{"cmdline":"s6-supervise s6rc-fdholder","cpu":"0.0","cpu_average":"0","mem":"0.0"},"29":{"cmdline":"s6-supervise go2rtc-healthcheck","cpu":"0.0","cpu_average":"0","mem":"0.0"},"30":{"cmdline":"s6-supervise frigate-log","cpu":"0.0","cpu_average":"0","mem":"0.0"},"31":{"cmdline":"s6-supervise s6rc-oneshot-runner","cpu":"0.0","cpu_average":"0","mem":"0.0"},"32":{"cmdline":"s6-supervise go2rtc","cpu":"0.0","cpu_average":"0","mem":"0.0"},"43":{"cmdline":"/package/admin/s6-2.11.3.2/command/s6-fdholderd -1 -i data/rules","cpu":"0.0","cpu_average":"0","mem":"0.0"},"44":{"cmdline":"/package/admin/s6/command/s6-ipcserverd -1 -- /package/admin/s6/command/s6-ipcserver-access -v0 -E -l0 -i data/rules -- /package/admin/s6/command/s6-sudod -t 30000 -- /package/admin/s6-rc/command/s6-rc-oneshot-run -l ../.. --","cpu":"0.0","cpu_average":"0","mem":"0.0"},"88":{"cmdline":"s6-log -b -- T 1 n0 s10000000 T /dev/shm/logs/nginx","cpu":"0.0","cpu_average":"0","mem":"0.0"},"89":{"cmdline":"s6-log -b -- T 1 n0 s10000000 T /dev/shm/logs/go2rtc","cpu":"0.0","cpu_average":"0","mem":"0.0"},"91":{"cmdline":"s6-log -b -- T 1 n0 s10000000 T /dev/shm/logs/frigate","cpu":"0.0","cpu_average":"0","mem":"0.0"},"98":{"cmdline":"/usr/local/go2rtc/bin/go2rtc -config=/dev/shm/go2rtc.yaml","cpu":"14.9","cpu_average":"20","mem":"0.3"},"109":{"cmdline":"python3 -u -m frigate","cpu":"12.4","cpu_average":"3","mem":"4.6"},"121":{"cmdline":"nginx: master process nginx","cpu":"0.0","cpu_average":"0","mem":"0.0"},"122":{"cmdline":"bash ./run.user go2rtc-healthcheck","cpu":"0.0","cpu_average":"0","mem":"0.0"},"151":{"cmdline":"nginx: worker process","cpu":"0.0","cpu_average":"0","mem":"0.0"},"152":{"cmdline":"nginx: worker process","cpu":"0.0","cpu_average":"0","mem":"0.0"},"153":{"cmdline":"nginx: worker process","cpu":"0.0","cpu_average":"0","mem":"0.0"},"154":{"cmdline":"nginx: worker process","cpu":"0.0","cpu_average":"0","mem":"0.0"},"155":{"cmdline":"nginx: worker process","cpu":"0.0","cpu_average":"0","mem":"0.0"},"158":{"cmdline":"nginx: worker process","cpu":"0.0","cpu_average":"0","mem":"0.0"},"380":{"cmdline":"frigate.logger       ","cpu":"0.0","cpu_average":"0","mem":"0.7"},"388":{"cmdline":"frigate.recording_manager","cpu":"0.0","cpu_average":"3","mem":"1.6"},"397":{"cmdline":"/usr/bin/python3 -c from multiprocessing.resource_tracker import main;main(59)","cpu":"0.0","cpu_average":"0","mem":"0.0"},"398":{"cmdline":"frigate.detector.tensorrt","cpu":"101.5","cpu_average":"88","mem":"3.4"},"400":{"cmdline":"frigate.output       ","cpu":"2.5","cpu_average":"1","mem":"0.9"},"406":{"cmdline":"ffmpeg -f rawvideo -pix_fmt yuv420p -video_size 1920x1080 -i pipe: -f mpegts -s 1280x720 -codec:v mpeg1video -q 8 -bf 0 pipe:","cpu":"0.0","cpu_average":"0","mem":"0.0"},"407":{"cmdline":"frigate.process:chicken","cpu":"0.0","cpu_average":"0","mem":"0.9"},"408":{"cmdline":"frigate.process:front_door","cpu":"5.0","cpu_average":"2","mem":"1.0"},"410":{"cmdline":"frigate.process:front_driveway","cpu":"2.5","cpu_average":"0","mem":"1.1"},"412":{"cmdline":"ffmpeg -f rawvideo -pix_fmt yuv420p -video_size 640x480 -i pipe: -f mpegts -s 960x720 -codec:v mpeg1video -q 8 -bf 0 pipe:","cpu":"0.0","cpu_average":"0","mem":"0.0"},"413":{"cmdline":"frigate.process:front_porch","cpu":"0.0","cpu_average":"1","mem":"1.0"},"416":{"cmdline":"frigate.process:garage","cpu":"0.0","cpu_average":"1","mem":"1.0"},"417":{"cmdline":"ffmpeg -f rawvideo -pix_fmt yuv420p -video_size 1280x720 -i pipe: -f mpegts -s 1280x720 -codec:v mpeg1video -q 8 -bf 0 pipe:","cpu":"0.0","cpu_average":"0","mem":"0.0"},"420":{"cmdline":"frigate.process:garage_se","cpu":"2.5","cpu_average":"1","mem":"1.0"},"422":{"cmdline":"frigate.process:garage_sw","cpu":"2.5","cpu_average":"0","mem":"1.0"},"423":{"cmdline":"ffmpeg -f rawvideo -pix_fmt yuv420p -video_size 640x480 -i pipe: -f mpegts -s 960x720 -codec:v mpeg1video -q 8 -bf 0 pipe:","cpu":"0.0","cpu_average":"0","mem":"0.0"},"425":{"cmdline":"frigate.process:mud_room","cpu":"0.0","cpu_average":"1","mem":"1.0"},"426":{"cmdline":"frigate.process:nc400","cpu":"2.5","cpu_average":"1","mem":"1.0"},"429":{"cmdline":"ffmpeg -f rawvideo -pix_fmt yuv420p -video_size 640x480 -i pipe: -f mpegts -s 960x720 -codec:v mpeg1video -q 8 -bf 0 pipe:","cpu":"0.0","cpu_average":"0","mem":"0.0"},"430":{"cmdline":"frigate.process:pole_barn_porch","cpu":"2.5","cpu_average":"0","mem":"1.1"},"431":{"cmdline":"frigate.process:road ","cpu":"2.5","cpu_average":"0","mem":"1.1"},"433":{"cmdline":"ffmpeg -f rawvideo -pix_fmt yuv420p -video_size 896x672 -i pipe: -f mpegts -s 960x720 -codec:v mpeg1video -q 8 -bf 0 pipe:","cpu":"0.0","cpu_average":"0","mem":"0.0"},"434":{"cmdline":"frigate.capture:chicken","cpu":"2.5","cpu_average":"1","mem":"0.9"},"436":{"cmdline":"ffmpeg -f rawvideo -pix_fmt yuv420p -video_size 896x672 -i pipe: -f mpegts -s 960x720 -codec:v mpeg1video -q 8 -bf 0 pipe:","cpu":"0.0","cpu_average":"0","mem":"0.0"},"437":{"cmdline":"frigate.capture:front_door","cpu":"2.5","cpu_average":"2","mem":"0.9"},"442":{"cmdline":"frigate.capture:front_driveway","cpu":"2.5","cpu_average":"1","mem":"0.9"},"443":{"cmdline":"frigate.capture:front_porch","cpu":"0.0","cpu_average":"0","mem":"0.9"},"444":{"cmdline":"ffmpeg -hide_banner -loglevel warning -threads 2 -hwaccel cuda -hwaccel_output_format cuda -user_agent FFmpeg Frigate/0.13.0-9185753 -rtsp_transport tcp -timeout 5000000 -fflags nobuffer -flags low_delay -i rtsp://127.0.0.1:8554/chicken -f segment -segment_time 10 -segment_format mp4 -reset_timestamps 1 -strftime 1 -c:v copy -c:a aac /tmp/cache/chicken-%Y%m%d%H%M%S.mp4 -r 5 -vf fps=5,scale_cuda=w=1920:h=1080:format=nv12,hwdownload,format=nv12,format=yuv420p -threads 2 -f rawvideo -pix_fmt yuv420p pipe:","cpu":"2.5","cpu_average":"3","mem":"2.4"},"447":{"cmdline":"frigate.capture:garage","cpu":"0.0","cpu_average":"0","mem":"0.9"},"449":{"cmdline":"ffmpeg -hide_banner -loglevel warning -threads 2 -hwaccel cuda -hwaccel_output_format cuda -user_agent FFmpeg Frigate/0.13.0-9185753 -rtsp_transport tcp -timeout 5000000 -fflags nobuffer -flags low_delay -i rtsp://127.0.0.1:8554/front_driveway_sub -r 10 -vf fps=10,scale_cuda=w=1280:h=720:format=nv12,hwdownload,format=nv12,format=yuv420p -threads 2 -f rawvideo -pix_fmt yuv420p pipe:","cpu":"7.4","cpu_average":"5","mem":"2.2"},"450":{"cmdline":"frigate.capture:garage_se","cpu":"2.5","cpu_average":"0","mem":"0.9"},"455":{"cmdline":"ffmpeg -f rawvideo -pix_fmt yuv420p -video_size 640x480 -i pipe: -f mpegts -s 960x720 -codec:v mpeg1video -q 8 -bf 0 pipe:","cpu":"0.0","cpu_average":"0","mem":"0.0"},"456":{"cmdline":"frigate.capture:garage_sw","cpu":"0.0","cpu_average":"0","mem":"1.0"},"459":{"cmdline":"ffmpeg -hide_banner -loglevel warning -threads 2 -hwaccel cuda -hwaccel_output_format cuda -user_agent FFmpeg Frigate/0.13.0-9185753 -rtsp_transport tcp -timeout 5000000 -fflags nobuffer -flags low_delay -i rtsp://127.0.0.1:8554/garage_sub -r 10 -vf fps=10,scale_cuda=w=640:h=480:format=nv12,hwdownload,format=nv12,format=yuv420p -threads 2 -f rawvideo -pix_fmt yuv420p pipe:","cpu":"2.5","cpu_average":"2","mem":"2.0"},"461":{"cmdline":"frigate.capture:mud_room","cpu":"0.0","cpu_average":"0","mem":"0.9"},"468":{"cmdline":"ffmpeg -hide_banner -loglevel warning -threads 2 -hwaccel cuda -hwaccel_output_format cuda -user_agent FFmpeg Frigate/0.13.0-9185753 -rtsp_transport tcp -timeout 5000000 -fflags nobuffer -flags low_delay -i rtsp://127.0.0.1:8554/front_porch_sub -r 10 -vf fps=10,scale_cuda=w=640:h=480:format=nv12,hwdownload,format=nv12,format=yuv420p -threads 2 -f rawvideo -pix_fmt yuv420p pipe:","cpu":"2.5","cpu_average":"2","mem":"2.1"},"469":{"cmdline":"frigate.capture:nc400","cpu":"2.5","cpu_average":"0","mem":"0.9"},"472":{"cmdline":"frigate.capture:pole_barn_porch","cpu":"0.0","cpu_average":"1","mem":"0.9"},"475":{"cmdline":"ffmpeg -f rawvideo -pix_fmt yuv420p -video_size 640x360 -i pipe: -f mpegts -s 1280x720 -codec:v mpeg1video -q 8 -bf 0 pipe:","cpu":"0.0","cpu_average":"0","mem":"0.0"},"477":{"cmdline":"frigate.capture:road ","cpu":"0.0","cpu_average":"1","mem":"0.9"},"478":{"cmdline":"ffmpeg -hide_banner -loglevel warning -threads 2 -hwaccel cuda -hwaccel_output_format cuda -user_agent FFmpeg Frigate/0.13.0-9185753 -rtsp_transport tcp -timeout 5000000 -fflags nobuffer -flags low_delay -i rtsp://127.0.0.1:8554/front_driveway -f segment -segment_time 10 -segment_format mp4 -reset_timestamps 1 -strftime 1 -c:v copy -c:a aac /tmp/cache/front_driveway-%Y%m%d%H%M%S.mp4","cpu":"0.0","cpu_average":"1","mem":"0.1"},"483":{"cmdline":"ffmpeg -hide_banner -loglevel warning -threads 2 -hwaccel cuda -hwaccel_output_format cuda -user_agent FFmpeg Frigate/0.13.0-9185753 -rtsp_transport tcp -timeout 5000000 -fflags nobuffer -flags low_delay -i rtsp://127.0.0.1:8554/mud_room_sub -r 10 -vf fps=10,scale_cuda=w=640:h=480:format=nv12,hwdownload,format=nv12,format=yuv420p -threads 2 -f rawvideo -pix_fmt yuv420p pipe:","cpu":"0.0","cpu_average":"2","mem":"2.1"},"486":{"cmdline":"ffmpeg -hide_banner -loglevel warning -threads 2 -hwaccel cuda -hwaccel_output_format cuda -user_agent FFmpeg Frigate/0.13.0-9185753 -rtsp_transport tcp -timeout 5000000 -fflags nobuffer -flags low_delay -i rtsp://127.0.0.1:8554/garage -f segment -segment_time 10 -segment_format mp4 -reset_timestamps 1 -strftime 1 -c:v copy -c:a aac /tmp/cache/garage-%Y%m%d%H%M%S.mp4","cpu":"2.5","cpu_average":"2","mem":"0.1"},"493":{"cmdline":"ffmpeg -hide_banner -loglevel warning -threads 2 -hwaccel cuda -hwaccel_output_format cuda -user_agent FFmpeg Frigate/0.13.0-9185753 -rtsp_transport tcp -timeout 5000000 -fflags nobuffer -flags low_delay -i rtsp://127.0.0.1:8554/front_porch -f segment -segment_time 10 -segment_format mp4 -reset_timestamps 1 -strftime 1 -c:v copy -c:a aac /tmp/cache/front_porch-%Y%m%d%H%M%S.mp4","cpu":"0.0","cpu_average":"1","mem":"0.1"},"501":{"cmdline":"ffmpeg -f rawvideo -pix_fmt yuv420p -video_size 1280x720 -i pipe: -f mpegts -s 1280x720 -codec:v mpeg1video -q 8 -bf 0 pipe:","cpu":"0.0","cpu_average":"0","mem":"0.0"},"502":{"cmdline":"ffmpeg -hide_banner -loglevel warning -threads 2 -hwaccel cuda -hwaccel_output_format cuda -user_agent FFmpeg Frigate/0.13.0-9185753 -rtsp_transport tcp -timeout 5000000 -fflags nobuffer -flags low_delay -i rtsp://127.0.0.1:8554/mud_room -f segment -segment_time 10 -segment_format mp4 -reset_timestamps 1 -strftime 1 -c:v copy -c:a aac /tmp/cache/mud_room-%Y%m%d%H%M%S.mp4","cpu":"0.0","cpu_average":"0","mem":"0.1"},"503":{"cmdline":"ffmpeg -hide_banner -loglevel warning -threads 2 -hwaccel cuda -hwaccel_output_format cuda -user_agent FFmpeg Frigate/0.13.0-9185753 -rtsp_transport tcp -timeout 5000000 -fflags nobuffer -flags low_delay -i rtsp://127.0.0.1:8554/nc400_sub -r 10 -vf fps=10,scale_cuda=w=640:h=360:format=nv12,hwdownload,format=nv12,format=yuv420p -threads 2 -f rawvideo -pix_fmt yuv420p pipe:","cpu":"0.0","cpu_average":"1","mem":"1.9"},"504":{"cmdline":"ffmpeg -hide_banner -loglevel warning -threads 2 -hwaccel cuda -hwaccel_output_format cuda -user_agent FFmpeg Frigate/0.13.0-9185753 -rtsp_transport tcp -timeout 5000000 -fflags nobuffer -flags low_delay -i rtsp://127.0.0.1:8554/pole_barn_porch_sub -r 10 -vf fps=10,scale_cuda=w=1280:h=720:format=nv12,hwdownload,format=nv12,format=yuv420p -threads 2 -f rawvideo -pix_fmt yuv420p pipe:","cpu":"5.0","cpu_average":"5","mem":"2.0"},"505":{"cmdline":"ffmpeg -hide_banner -loglevel warning -threads 2 -hwaccel cuda -hwaccel_output_format cuda -user_agent FFmpeg Frigate/0.13.0-9185753 -rtsp_transport tcp -timeout 5000000 -fflags nobuffer -flags low_delay -i rtsp://127.0.0.1:8554/road -r 10 -vf fps=10,scale_cuda=w=1280:h=720:format=nv12,hwdownload,format=nv12,format=yuv420p -threads 2 -f rawvideo -pix_fmt yuv420p pipe:","cpu":"5.0","cpu_average":"5","mem":"2.2"},"508":{"cmdline":"ffmpeg -f rawvideo -pix_fmt yuv420p -video_size 1280x720 -i pipe: -f mpegts -s 1280x720 -codec:v mpeg1video -q 8 -bf 0 pipe:","cpu":"0.0","cpu_average":"0","mem":"0.0"},"509":{"cmdline":"ffmpeg -hide_banner -loglevel warning -threads 2 -hwaccel cuda -hwaccel_output_format cuda -user_agent FFmpeg Frigate/0.13.0-9185753 -rtsp_transport tcp -timeout 5000000 -fflags nobuffer -flags low_delay -i rtsp://127.0.0.1:8554/pole_barn_porch -f segment -segment_time 10 -segment_format mp4 -reset_timestamps 1 -strftime 1 -c:v copy -c:a aac /tmp/cache/pole_barn_porch-%Y%m%d%H%M%S.mp4","cpu":"0.0","cpu_average":"1","mem":"0.1"},"513":{"cmdline":"ffmpeg -f rawvideo -pix_fmt yuv420p -video_size 1280x720 -i pipe: -f mpegts -s 1280x720 -codec:v mpeg1video -q 8 -bf 0 pipe:","cpu":"0.0","cpu_average":"0","mem":"0.0"},"514":{"cmdline":"ffmpeg -hide_banner -loglevel warning -threads 2 -hwaccel cuda -hwaccel_output_format cuda -user_agent FFmpeg Frigate/0.13.0-9185753 -rtsp_transport tcp -timeout 5000000 -fflags nobuffer -flags low_delay -i rtsp://127.0.0.1:8554/nc400 -f segment -segment_time 10 -segment_format mp4 -reset_timestamps 1 -strftime 1 -c:v copy -c:a aac /tmp/cache/nc400-%Y%m%d%H%M%S.mp4","cpu":"0.0","cpu_average":"0","mem":"0.1"},"105312":{"cmdline":"ffmpeg -hide_banner -loglevel warning -threads 2 -hwaccel cuda -hwaccel_output_format cuda -user_agent FFmpeg Frigate/0.13.0-9185753 -rtsp_transport tcp -timeout 5000000 -fflags nobuffer -flags low_delay -i rtsp://127.0.0.1:8554/front_door_sub -r 30 -vf fps=30,scale_cuda=w=640:h=480:format=nv12,hwdownload,format=nv12,format=yuv420p -threads 2 -f rawvideo -pix_fmt yuv420p pipe:","cpu":"5.0","cpu_average":"5","mem":"2.0"},"105314":{"cmdline":"ffmpeg -hide_banner -loglevel warning -threads 2 -hwaccel cuda -hwaccel_output_format cuda -user_agent FFmpeg Frigate/0.13.0-9185753 -rtsp_transport tcp -timeout 5000000 -fflags nobuffer -flags low_delay -i rtsp://127.0.0.1:8554/front_door -f segment -segment_time 10 -segment_format mp4 -reset_timestamps 1 -strftime 1 -c:v copy -c:a aac /tmp/cache/front_door-%Y%m%d%H%M%S.mp4","cpu":"0.0","cpu_average":"1","mem":"0.3"},"182750":{"cmdline":"ffmpeg -hide_banner -loglevel warning -threads 2 -hwaccel cuda -hwaccel_output_format cuda -user_agent FFmpeg Frigate/0.13.0-9185753 -rtsp_transport tcp -timeout 5000000 -fflags nobuffer -flags low_delay -i rtsp://127.0.0.1:8554/garage_se_sub -r 7 -vf fps=7,scale_cuda=w=896:h=672:format=nv12,hwdownload,format=nv12,format=yuv420p -threads 2 -f rawvideo -pix_fmt yuv420p pipe:","cpu":"0.0","cpu_average":"1","mem":"2.1"},"184190":{"cmdline":"ffmpeg -hide_banner -loglevel warning -threads 2 -hwaccel cuda -hwaccel_output_format cuda -user_agent FFmpeg Frigate/0.13.0-9185753 -rtsp_transport tcp -timeout 5000000 -fflags nobuffer -flags low_delay -i rtsp://127.0.0.1:8554/garage_se -f segment -segment_time 10 -segment_format mp4 -reset_timestamps 1 -strftime 1 -c:v copy -c:a aac /tmp/cache/garage_se-%Y%m%d%H%M%S.mp4","cpu":"0.0","cpu_average":"1","mem":"0.3"},"190655":{"cmdline":"ffmpeg -hide_banner -loglevel warning -threads 2 -hwaccel cuda -hwaccel_output_format cuda -user_agent FFmpeg Frigate/0.13.0-9185753 -rtsp_transport tcp -timeout 5000000 -fflags nobuffer -flags low_delay -i rtsp://127.0.0.1:8554/garage_sw_sub -r 7 -vf fps=7,scale_cuda=w=896:h=672:format=nv12,hwdownload,format=nv12,format=yuv420p -threads 2 -f rawvideo -pix_fmt yuv420p pipe:","cpu":"2.5","cpu_average":"1","mem":"2.1"},"192407":{"cmdline":"ffmpeg -hide_banner -loglevel warning -threads 2 -hwaccel cuda -hwaccel_output_format cuda -user_agent FFmpeg Frigate/0.13.0-9185753 -rtsp_transport tcp -timeout 5000000 -fflags nobuffer -flags low_delay -i rtsp://127.0.0.1:8554/garage_sw -f segment -segment_time 10 -segment_format mp4 -reset_timestamps 1 -strftime 1 -c:v copy -c:a aac /tmp/cache/garage_sw-%Y%m%d%H%M%S.mp4","cpu":"0.0","cpu_average":"0","mem":"0.4"},"205514":{"cmdline":"sleep 30s","cpu":"0.0","cpu_average":"0","mem":"0.0"}},"detection_fps":16.2,"detectors":{"tensorrt":{"detection_start":0.0,"inference_speed":60.96,"pid":398}},"front_door":{"camera_fps":30.0,"capture_pid":437,"detection_enabled":1,"detection_fps":0.2,"ffmpeg_pid":105312,"pid":408,"process_fps":28.6,"skipped_fps":1.7},"front_driveway":{"camera_fps":10.0,"capture_pid":442,"detection_enabled":1,"detection_fps":1.5,"ffmpeg_pid":449,"pid":410,"process_fps":8.3,"skipped_fps":2.0},"front_porch":{"camera_fps":10.0,"capture_pid":443,"detection_enabled":1,"detection_fps":0.0,"ffmpeg_pid":468,"pid":413,"process_fps":10.0,"skipped_fps":0.0},"garage":{"camera_fps":10.0,"capture_pid":447,"detection_enabled":1,"detection_fps":0.0,"ffmpeg_pid":459,"pid":416,"process_fps":10.0,"skipped_fps":0.0},"garage_se":{"camera_fps":7.1,"capture_pid":450,"detection_enabled":1,"detection_fps":2.9,"ffmpeg_pid":182750,"pid":420,"process_fps":3.5,"skipped_fps":3.9},"garage_sw":{"camera_fps":7.0,"capture_pid":456,"detection_enabled":1,"detection_fps":1.4,"ffmpeg_pid":190655,"pid":422,"process_fps":5.8,"skipped_fps":1.5},"gpu_usages":{"NVIDIA GeForce GT 1030":{"gpu":"98.0%","mem":"49.48%"}},"mud_room":{"camera_fps":10.1,"capture_pid":461,"detection_enabled":1,"detection_fps":0.0,"ffmpeg_pid":483,"pid":425,"process_fps":10.1,"skipped_fps":0.0},"nc400":{"camera_fps":10.0,"capture_pid":469,"detection_enabled":1,"detection_fps":0.1,"ffmpeg_pid":503,"pid":426,"process_fps":10.0,"skipped_fps":0.0},"pole_barn_porch":{"camera_fps":10.0,"capture_pid":472,"detection_enabled":1,"detection_fps":5.0,"ffmpeg_pid":504,"pid":430,"process_fps":0.8,"skipped_fps":9.2},"processes":{"go2rtc":{"pid":98},"logger":{"pid":380},"recording":{"pid":388}},"road":{"camera_fps":10.1,"capture_pid":477,"detection_enabled":1,"detection_fps":5.1,"ffmpeg_pid":505,"pid":431,"process_fps":0.7,"skipped_fps":9.4},"service":{"last_updated":1694724937,"latest_version":"0.12.1","storage":{"/dev/shm":{"free":230.9,"mount_type":"tmpfs","total":256.0,"used":25.1},"/media/frigate/clips":{"free":453326.3,"mount_type":"btrfs","total":1001571.0,"used":547389.0},"/media/frigate/recordings":{"free":453326.3,"mount_type":"btrfs","total":1001571.0,"used":547389.0},"/tmp/cache":{"free":895.6,"mount_type":"tmpfs","total":953.7,"used":58.1}},"temperatures":{},"uptime":28359,"version":"0.13.0-9185753"}}

Operating system

Debian

Install method

Docker Compose

Coral version

CPU (no coral)

Network connection

Wired

Camera make and model

Mostly HikVision

Any other information that may be helpful

No response

NickM-27 commented 1 year ago

Without logs it is difficult to track this down, usually that means the GPU is failing to keep up and frames are being delivered slowly. Can you send a picture of the system page while this is happening?

jonadis commented 1 year ago

I bet it could be just that simple. I'm trying TensorRT and when things are fired up using TensorRT the GPU usage shows 97%-98% most of the time. I think I saw this same behavior when using OpenVino too but I'm not 100% sure. I'll switch back to OV and see if it improves. image

NickM-27 commented 1 year ago

oh, I see you are running detect at 30 fps on that camera, that is highly discouraged. The fps does not need to match the cameras fps

jonadis commented 1 year ago

It's a doorbell that I can't adjust the output stream settings and I thought the detect FPS had to match. I'll change that, however that camera is not one of the ones that often has this symptom.

NickM-27 commented 1 year ago

however that camera is not one of the ones that often has this symptom.

right, but it may be using many resources and frames that other cameras are trying to use

jonadis commented 1 year ago

Understood, I'll lower them all down to 5fps and see how it goes.

NickM-27 commented 1 year ago

worth noting that if you have an integrated GPU there is no reason one GPU has to be used for everything. For example I use my integrated GPU for the streams that I don't care about latency on and then use my nvidia GPU for the streams that I want the lowest detection latency possible

jonadis commented 1 year ago

Good to know. I did fire things up with detect on 5FPS for all cameras and 2/3 of the cameras had the no frames message... I switched back to OpenVino and the issue seems gone so far. I do have a Coral but its a PCI Express one and this motherboard doesn't have enough PCI Express slots to hold both the GPU and the Coral, so I've planning to replace the motherboard.... Guess I'll fasttrack that now. Thanks for all your help.

NickM-27 commented 1 year ago

this issue seems to have been tracked down and the behavior is expected given system page so will close, feel free to create a new issue if something else comes up