search

blakeblackshear / frigate

NVR with realtime local object detection for IP cameras
https://frigate.video
MIT License
18.41k stars 1.68k forks source link

[Detector Support]: High CPU-Usage with CoralTPU #9018

Closed BlackRapsbeere closed 8 months ago

BlackRapsbeere commented 9 months ago

Describe the problem you are having

Hi everyone. I am kind of concerned about the CPU usage of my detect process for the doorbell. Please see attached screenshot. I have set the detect fps to 10 and a coral via M.2. I do not understand how there is such high CPU usage. It is an Intel Xeon E3-1245 v5 and I hope the 158% are CPU-time in thousand miliseconds. The Intel QSV is working as indicated by the usage. Also the TPU seems to work. Where is the fault? Appreciate all the ideas and help! Regards grafik

Version

0.12.1-367d724

Frigate config file

birdseye:
  enabled: true
  height: 1080
  mode: continuous
  quality: 8
  width: 1920
go2rtc:
  api:
    listen: ":1984"
    origin: '*'
  ffmpeg:
    bin: "ffmpeg"
    global: "-hide_banner"
    file: "-re -stream_loop -1 -i {input}"
    http: "-fflags nobuffer -flags low_delay -i {input}"
    rtsp: "-avoid_negative_ts make_zero -fflags +genpts+discardcorrupt -timeout 5000000 -use_wallclock_as_timestamps 1 -user_agent go2rtc/ffmpeg -rtsp_transport tcp -i {input}"
    output: "-user_agent ffmpeg/go2rtc -rtsp_transport tcp -f rtsp {output}"
    # ... different presets for codecs
  hass:
    config: ""
  log:
    format: text
    # log-levels: trace, debug, info, warn, error, fatal
    level: info  # default level
    api: info
    exec: info
    ngrok: info
    rtsp: info
    streams: info
    webrtc: info
  rtsp:
    listen: ":8554"
  srtp:
    listen: ":8443"
  webrtc:
    listen: ":8555"
    candidates:
      - 192.168.16.35:8555
      - stun:8555
  streams:
    Klingel:
      - rtsp://UserName1:PassWord1@192.168.16.103/cam/realmonitor?channel=1&subtype=0#backchannel=0
    KlingelSub:
      - rtsp://UserName1:PassWord1@192.168.16.103/cam/realmonitor?channel=1&subtype=1#backchannel=0
    KlingelAudio:
      - rtsp://UserName1:PassWord1@192.168.16.103/cam/realmonitor?channel=1&subtype=1
    FI9900P:
      - rtsp://UserName2:PassWord2@192.168.16.102:88/videoMain
    FI9900PSub:
      - rtsp://UserName2:PassWord2@192.168.16.102:88/videoSub
cameras:
  Klingel:
    enabled: true
    name: Klingel
    best_image_timeout: 60
    birdseye:
      enabled: true
      mode: continuous
    ffmpeg:
      inputs:
        - path: rtsp://127.0.0.1:8554/KlingelSub
          hwaccel_args: preset-intel-qsv-h264
          roles:
            - detect
        - path: rtsp://127.0.0.1:8554/Klingel
          hwaccel_args: preset-intel-qsv-h265
          roles:
            - record
      input_args: preset-rtsp-restream
      output_args:
        record: preset-record-generic-audio-copy
    live:
      stream_name: KlingelSub
      height: 576
      quality: 8
    ui:
      dashboard: true
      order: 1
    detect:
      enabled: true
      fps: 10
      width: 720
      height: 576
      max_disappeared: 100
      stationary:
        interval: 100
        max_frames:
          objects: {}
        threshold: 300
    record:
      enabled: true
      events:
        objects:
          - person
          - bicycle
          - car
          - motorcycle
          - bus
          - car
          - bird
          - cat
          - dog
          - hat
          - backpack
          - umbrella
          - eye glasses
          - handbag
          - suitcase
        pre_capture: 30
        post_capture: 20
        required_zones:
          - Vorgarten
        retain:
          default: 30
          mode: motion
          objects: {}
      expire_interval: 60
      retain:
        days: 0
        mode: all
    snapshots:
      bounding_box: false
      clean_copy: true
      crop: false
      enabled: true
      quality: 70
      required_zones: []
      retain:
        default: 30
        mode: motion
        objects: {}
      timestamp: true
    motion:
      threshold: 25
      contour_area: 25
      delta_alpha: 0.2
      frame_alpha: 0.2
      frame_height: 50
      improve_contrast: false
      mqtt_off_delay: 20
      mask:
        - 720,257,307,271,331,576,274,576,244,0,720,0,720,0
    zones:
      Vorgarten:
        coordinates: 464,362,720,448,720,576,334,576,320,381
    objects:
      filters:
        bicycle:
          mask:
            - 310,270,720,237,720,0,0,0,0,576,331,576
        bus:
          mask:
            - 310,270,720,237,720,0,0,0,0,576,331,576
        car:
          mask:
            - 310,270,720,237,720,0,0,0,0,576,331,576
        cat:
          mask:
            - 310,270,720,237,720,0,0,0,0,576,331,576
        dog:
          mask:
            - 310,270,720,237,720,0,0,0,0,576,331,576
        motorcycle:
          mask:
            - 310,270,720,237,720,0,0,0,0,576,331,576
    mqtt:
      bounding_box: true
      crop: true
      enabled: true
      height: 480
      quality: 80
      required_zones: []
      timestamp: true
    rtmp:
      enabled: false
    timestamp_style:
      color:
        blue: 255
        green: 255
        red: 255
      format: '%Y-%m-%d %H:%M:%S'
      position: tl
      thickness: 2
  FoscamFI9900P:
    enabled: true
    name: FoscamFI9900P
    best_image_timeout: 60
    birdseye:
      enabled: true
      mode: continuous
    ffmpeg:
      global_args:
        - '-hide_banner'
        - '-loglevel'
        - warning
      input_args: >-
        -avoid_negative_ts make_zero -fflags +genpts+discardcorrupt
        -rtsp_transport tcp -timeout 5000000 -use_wallclock_as_timestamps 1
      inputs:
        #- path: rtsp://UserName2:PassWord2@192.168.16.102:88/videoSub
        - path: rtsp://127.0.0.1:8554/FI9900PSub
          hwaccel_args: preset-intel-qsv-h264
          roles:
            - detect
        #- path: rtsp://UserName2:PassWord2@192.168.16.102:88/videoMain
        - path: rtsp://127.0.0.1:8554/FI9900P
          hwaccel_args: preset-intel-qsv-h264
          roles:
            - record
      output_args:
        detect:
          - '-f'
          - rawvideo
          - '-pix_fmt'
          - yuv420p
        record: >-
          -f segment
          -segment_time 10
          -segment_format mp4
          -reset_timestamps 1
          -strftime 1
          -c copy
          -an
    live:
      stream_name: FI9900PSub
      height: 720
      quality: 8
    ui:
      dashboard: true
      order: 0
    detect:
      enabled: true
      fps: 5
      width: 1280
      height: 720
      max_disappeared: 25
      stationary:
        interval: 10
        max_frames:
          objects: {}
        threshold: 50
    record:
      enabled: true
      events:
        objects:
          - person
          - bird
          - cat
          - dog
          - hat
          - backpack
          - umbrella
          - eye glasses
          - handbag
          - suitcase
        post_capture: 10
        pre_capture: 15
        required_zones: []
        retain:
          default: 30
          mode: motion
          objects: {}
      expire_interval: 60
      retain:
        days: 0
        mode: all
    snapshots:
      bounding_box: false
      clean_copy: true
      crop: false
      enabled: true
      quality: 70
      required_zones: []
      retain:
        default: 30
        mode: motion
        objects: {}
      timestamp: true
    motion:
      threshold: 25
      contour_area: 30
      delta_alpha: 0.2
      frame_alpha: 0.2
      frame_height: 50
      improve_contrast: false
      mask:
        - 0,720,368,720,201,373,71,0,0,0
      mqtt_off_delay: 20

    zones:
      Keller:
        coordinates: 328,720,585,671,323,334,188,389
      Terasse:
        coordinates: 574,239,570,22,470,0,73,0,209,368
      Weg:
        coordinates: 344,322,654,720,1174,720,1280,445,1280,283,1132,263,891,490,702,483,468,279
      Garten:
        coordinates: 578,0,1087,0,1280,55,1280,290,1126,264,885,491,699,485,462,274,579,243
    mqtt:
      bounding_box: true
      crop: true
      enabled: true
      height: 480
      quality: 80
      required_zones: []
      timestamp: true
    objects:
      filters:
        backpack:
          max_area: 24000000
          max_ratio: 24000000
          min_area: 0
          min_ratio: 0
          min_score: 0.5
          threshold: 0.7
        bird:
          max_area: 24000000
          max_ratio: 24000000
          min_area: 0
          min_ratio: 0
          min_score: 0.5
          threshold: 0.7
        cat:
          max_area: 24000000
          max_ratio: 24000000
          min_area: 0
          min_ratio: 0
          min_score: 0.5
          threshold: 0.7
        dog:
          max_area: 24000000
          max_ratio: 24000000
          min_area: 0
          min_ratio: 0
          min_score: 0.5
          threshold: 0.7
        eye glasses:
          max_area: 24000000
          max_ratio: 24000000
          min_area: 0
          min_ratio: 0
          min_score: 0.5
          threshold: 0.7
        handbag:
          max_area: 24000000
          max_ratio: 24000000
          min_area: 0
          min_ratio: 0
          min_score: 0.5
          threshold: 0.7
        hat:
          max_area: 24000000
          max_ratio: 24000000
          min_area: 0
          min_ratio: 0
          min_score: 0.5
          threshold: 0.7
        person:
          max_area: 350000
          max_ratio: 4
          min_area: 680
          min_ratio: 0.2
          min_score: 0.5
          threshold: 0.7
        suitcase:
          max_area: 24000000
          max_ratio: 24000000
          min_area: 0
          min_ratio: 0
          min_score: 0.5
          threshold: 0.7
        umbrella:
          max_area: 24000000
          max_ratio: 24000000
          min_area: 0
          min_ratio: 0
          min_score: 0.5
          threshold: 0.7
      mask: ''
      track:
        - person
        - bird
        - cat
        - dog
        - hat
        - backpack
        - umbrella
        - eye glasses
        - handbag
        - suitcase
    rtmp:
      enabled: false
    timestamp_style:
      color:
        blue: 255
        green: 255
        red: 255
      format: '%Y-%m-%d %H:%M:%S'
      position: tl
      thickness: 2
detect:
  enabled: true
detectors:
  CoralTPU-PCIe:
    device: pci
    type: edgetpu
environment_vars: {}
ffmpeg:
  global_args:
    - '-hide_banner'
    - '-loglevel'
    - warning
  hwaccel_args: preset-vaapi
  input_args: >-
    -avoid_negative_ts make_zero -fflags +genpts+discardcorrupt -rtsp_transport
    tcp -timeout 5000000 -use_wallclock_as_timestamps 1
  output_args:
    detect:
      - '-f'
      - rawvideo
      - '-pix_fmt'
      - yuv420p
    record: >-
      -f segment -segment_time 10 -segment_format mp4 -reset_timestamps 1
      -strftime 1 -c copy
    rtmp: >-
      -avoid_negative_ts make_zero -fflags +genpts+discardcorrupt
      -rtsp_transport tcp -timeout 5000000 -use_wallclock_as_timestamps 1
logger:
  default: warning
  logs: {}
model:
  height: 320
  labelmap: {}
  width: 320
motion:
  contour_area: 50
  delta_alpha: 0.2
  frame_alpha: 0.2
  frame_height: 50
  improve_contrast: false
  mask: ''
  mqtt_off_delay: 20
  threshold: 25
objects:
  mask: ''
  track:
    - person
    - bicycle
    - car
    - motorcycle
    - bus
    - car
    - bird
    - cat
    - dog
    - hat
    - backpack
    - umbrella
    - eye glasses
    - handbag
    - suitcase
record:
  enabled: true
  events:
    objects:
      - person
      - bicycle
      - car
      - motorcycle
      - bus
      - car
      - bird
      - cat
      - dog
      - hat
      - backpack
      - umbrella
      - eye glasses
      - handbag
      - suitcase
    post_capture: 10
    pre_capture: 10
    required_zones: []
    retain:
      default: 30
      mode: motion
      objects: {}
  expire_interval: 60
  retain:
    days: 0
    mode: all
rtmp:
  enabled: false
snapshots:
  bounding_box: false
  clean_copy: true
  crop: false
  enabled: true
  quality: 70
  required_zones: []
  retain:
    default: 30
    mode: motion
    objects: {}
  timestamp: true
timestamp_style:
  color:
    blue: 255
    green: 255
    red: 255
  format: '%Y-%m-%d %H:%M:%S'
  position: tl
  thickness: 2
ui:
  use_experimental: false
mqtt:
  client_id: frigate
  host: mosquitto.ix-mosquitto.svc.cluster.local
  port: 1883
  stats_interval: 60
  topic_prefix: frigate

docker-compose file or Docker CLI command

I am running the TrueCharts helm chart for TrueNAS Cobia.

Relevant log output

None.

Operating system

Other Linux

Install method

Docker CLI

Coral version

M.2

Any other information that may be helpful

No response

NickM-27 commented 9 months ago

for starts I would recommend using ffmpeg presets for input_args and record_args and I would also not recommend overriding detect or global args, you are removing args from the defaults that are there to reduce cpu usage

BlackRapsbeere commented 9 months ago

As far as I can see from the frigate-code on github, I did actually not change anything for the mentioned camera. All the setting are as they would be in the preset. I just rather see the settings explicitly. I added the threads now and will see how it is going tomorrow

NickM-27 commented 9 months ago

that is incorrect, the global args and detect args are outdated. Also, not using presets means that when we make additional improvements to the args they will not be automatically applied.

nwish commented 9 months ago

I doubt this affects CPU usage, but your max_ratio setting for most of your objects is astronomical. Is there a reason? Are those actually the default settings?

NickM-27 commented 9 months ago

Are those actually the default settings?

Those are the default values

nwish commented 9 months ago

Are those actually the default settings?

Those are the default values

Gotcha, I thought they might be.

BlackRapsbeere commented 9 months ago

So I got rid of the ffmpeg settings and returned to the presets. I also deleted the go2rtc settings for ffmpeg. the cpu usage of that one camera stays way to high around 100% Any other ideas?

NickM-27 commented 9 months ago

would want to check the debug live view with motion and object boxes enabled and see what is going on

BlackRapsbeere commented 9 months ago

I set the config as recommended. Nothing really changed except the dual-thread usage kind of smoothes things out and makes it more reliable. So I tried to watch the debug screens a little while. As soon as there is something moving in the image the CPU-usage goes up to about 50%. The debug screen show one or two "regions" covering most of the image. In this regions nothing is actually moving. There are 4 cars in 3 different drive-ways and one front door of a house is detected as a person. The region then stays there for about a minute, I guess, and then disapears. When there is no region shown, the CPU usage drops to about 1% Any further ideas?

github-actions[bot] commented 8 months ago

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.