search

blakeblackshear / frigate

NVR with realtime local object detection for IP cameras
https://frigate.video
MIT License
17.66k stars 1.62k forks source link

[Support]: Coral dev mini, high inference speed #2852

Closed WGrobler closed 2 years ago

WGrobler commented 2 years ago

Describe the problem you are having

I recently obtained a Coral Dev Mini to run Frigate. I know the Dev mini is not the most powerful compute module, but its the only thing I could get in stock. From what I could gather it uses the same TPU as the USB version, so it should be working fine. However after a lot of tinkering to get frigate to work on the board, I am not getting the performance I was expecting.

Looking at the docs I can see that the max detection framerate I can expect is 1000/(interference speed). Currently I am getting inference speed of 70-75, this in turn will give me a combined max of about 14 FPS.

I have no idea how powerful the coral TPUs actually are, but I was hoping to get at least 30 FPS from this device.

What is the expected detection framerate for coral TPUs? I know other factors will determine the final results, but at this time I don't know if the device is under performing or actually working at max performance.

Version

0.9.4-26AE608

Frigate config file

mqtt:
  host: 172.20.0.3
  port: 1883

detectors:
  coral:
    type: edgetpu
    device: usb

cameras:
  front:
    ffmpeg:
      inputs:
        - path: rtsp://user:password@192.168.0.20:554/PSIA/streaming/channels/302
          roles:
            - detect
    detect:
      width: 352
      height: 288
      fps: 10
    rtmp:
      enabled: False
    snapshots:
      enabled: True
      timestamp: True
      bounding_box: True
      height: 175
      retain:
        default: 10
        objects:
          person: 15
    motion:
      mask:
        - 134,0,0,0,0,55
        - 290,266,253,267,256,255,291,253
    zones:
      front_zone_0:
        coordinates: 0,288,0,134,66,91,186,45,193,0,352,0,352,288
        objects:
          - person
          - cat
      front_zone_1:
        coordinates: 0,181,214,64,216,0,144,0,0,42
        objects:
          - person
    objects:
      track:
        - person
        - cat

  garage:
    ffmpeg:
      inputs:
        - path: rtsp://user:password@192.168.0.20:554/PSIA/streaming/channels/102
          roles:
            - detect
    detect:
      width: 352
      height: 288
      fps: 10
    rtmp:
      enabled: False
    snapshots:
      enabled: True
      timestamp: True
      bounding_box: True
      height: 175
      retain:
        default: 10
        objects:
          person: 15
    objects:
      track:
        - person
        - car
        - motorcycle
    motion:
      mask:
        - 352,0,319,0,320,77,352,78

  gate:
    ffmpeg:
      inputs:
        - path: rtsp://user:password@192.168.0.20:554/PSIA/streaming/channels/202
          roles:
            - detect
    detect:
      width: 352
      height: 288
      fps: 10
    rtmp:
      enabled: False
    snapshots:
      enabled: True
      timestamp: True
      bounding_box: True
      height: 175
      required_zones: [gate_zone_0,gate_zone_1]
      retain:
        default: 10
        objects:
          person: 15
    objects:
      track:
        - person
        - car
        - motorcycle
      filters:
        car:
          mask: 318,22,352,21,352,0,214,0,0,0,0,288,215,112,214,75
        motorcycle:
          mask: 318,22,352,21,352,0,214,0,0,0,0,288,215,112,214,75
        person:
          mask: 318,22,352,21,352,0,214,0,0,0,0,288,215,112,214,75
    motion:
      mask:
        - 318,22,352,21,352,0,214,0,0,0,0,288,215,112,214,75
        - 352,27,331,25,331,79,352,70
    zones:
      gate_zone_0:
        coordinates: 235,56,245,201,191,288,352,288,352,74,331,73,331,27,317,21
        objects:
          - person
      gate_zone_1:
        coordinates: 233,121,38,288,190,288,239,210
        objects:
          - person
          - car
          - motorcycle

  back:
    ffmpeg:
      inputs:
        - path: rtsp://user:password@192.168.0.20:554/PSIA/streaming/channels/402
          roles:
            - detect
    detect:
      width: 352
      height: 288
      fps: 5
    rtmp:
      enabled: False
    snapshots:
      enabled: True
      timestamp: True
      bounding_box: True
      height: 175
      retain:
        default: 10
        objects:
          person: 15

    motion:
      mask:
        - 201,0,0,0,0,143
        - 168,288,203,224,352,224,352,288
        - 180,0,185,36,352,35,352,0

Relevant log output

frigate    | [services.d] starting services
frigate    | [services.d] done.
frigate    | [2022-02-21 13:48:09] frigate.app                    INFO    : Starting Frigate (0.9.4-26ae608)
frigate    | Starting migrations
frigate    | [2022-02-21 13:48:09] peewee_migrate                 INFO    : Starting migrations
frigate    | There is nothing to migrate
frigate    | [2022-02-21 13:48:09] peewee_migrate                 INFO    : There is nothing to migrate
frigate    | [2022-02-21 13:48:09] frigate.mqtt                   INFO    : MQTT connected
frigate    | [2022-02-21 13:48:09] frigate.app                    INFO    : Output process started: 217
frigate    | [2022-02-21 13:48:09] detector.coral                 INFO    : Starting detection process: 216
frigate    | [2022-02-21 13:48:09] frigate.edgetpu                INFO    : Attempting to load TPU as usb
frigate    | [2022-02-21 13:48:09] frigate.app                    INFO    : Camera processor started for front: 220
frigate    | [2022-02-21 13:48:09] ws4py                          INFO    : Using epoll
frigate    | [2022-02-21 13:48:09] frigate.app                    INFO    : Camera processor started for garage: 225
frigate    | [2022-02-21 13:48:09] frigate.app                    INFO    : Camera processor started for gate: 227
frigate    | [2022-02-21 13:48:09] frigate.app                    INFO    : Camera processor started for back: 229
frigate    | [2022-02-21 13:48:09] frigate.app                    INFO    : Capture process started for front: 231
frigate    | [2022-02-21 13:48:09] frigate.app                    INFO    : Capture process started for garage: 232
frigate    | [2022-02-21 13:48:09] frigate.app                    INFO    : Capture process started for gate: 235
frigate    | [2022-02-21 13:48:09] frigate.app                    INFO    : Capture process started for back: 238
frigate    | [2022-02-21 13:48:10] ws4py                          INFO    : Using epoll
frigate    | [2022-02-21 13:48:13] frigate.edgetpu                INFO    : TPU found

FFprobe output from your camera

ffprobe version 4.3.1 Copyright (c) 2007-2020 the FFmpeg developers
  built with gcc 9 (Ubuntu 9.3.0-17ubuntu1~20.04)
  configuration: --disable-debug --disable-doc --disable-ffplay --enable-shared --enable-avresample --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-gpl --enable-libfreetype --enable-libvidstab --enable-libmp3lame --enable-libopus --enable-libtheora --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libxcb --enable-libx265 --enable-libxvid --enable-libx264 --enable-nonfree --enable-openssl --enable-libfdk_aac --enable-postproc --enable-small --enable-version3 --enable-libzmq --extra-libs=-ldl --prefix=/opt/ffmpeg --enable-libopenjpeg --enable-libkvazaar --enable-libaom --extra-libs=-lpthread --enable-rkmpp --enable-libdrm --enable-v4l2_m2m --enable-neon --extra-cflags=-I/opt/ffmpeg/include --extra-ldflags=-L/opt/ffmpeg/lib
  libavutil      56. 51.100 / 56. 51.100
  libavcodec     58. 91.100 / 58. 91.100
  libavformat    58. 45.100 / 58. 45.100
  libavdevice    58. 10.100 / 58. 10.100
  libavfilter     7. 85.100 /  7. 85.100
  libavresample   4.  0.  0 /  4.  0.  0
  libswscale      5.  7.100 /  5.  7.100
  libswresample   3.  7.100 /  3.  7.100
  libpostproc    55.  7.100 / 55.  7.100
Input #0, rtsp, from 'rtsp://user:password@192.168.0.20:554/PSIA/streaming/channels/102':
  Metadata:
    title           : Media Server V3.4.84
    comment         : Media Server Session Description : standard
  Duration: N/A, start: 0.660000, bitrate: N/A
    Stream #0:0: Video: h264, yuv420p(progressive), 352x288, 10 fps, 25 tbr, 90k tbn, 20 tbc

Input #0, rtsp, from 'rtsp://user:password@192.168.0.20:554/PSIA/streaming/channels/202':
  Metadata:
    title           : Media Server V3.4.84
    comment         : Media Server Session Description : standard
  Duration: N/A, start: 0.621000, bitrate: N/A
    Stream #0:0: Video: h264, yuv420p(progressive), 352x288, 10 fps, 25 tbr, 90k tbn, 20 tbc

Input #0, rtsp, from 'rtsp://user:password@192.168.0.20:554/PSIA/streaming/channels/302':
  Metadata:
    title           : Media Server V3.4.84
    comment         : Media Server Session Description : standard
  Duration: N/A, start: 0.619000, bitrate: N/A
    Stream #0:0: Video: h264, yuv420p(progressive), 352x288, 10 fps, 25 tbr, 90k tbn, 20 tbc

Input #0, rtsp, from 'rtsp://user:password@192.168.0.20:554/PSIA/streaming/channels/402':
  Metadata:
    title           : Media Server V3.4.84
    comment         : Media Server Session Description : standard
  Duration: N/A, start: 1.510000, bitrate: N/A
    Stream #0:0: Video: h264, yuv420p(progressive), 352x288, 4 fps, 25 tbr, 90k tbn, 8 tbc

Frigate stats

{"back":{"camera_fps":5.1,"capture_pid":238,"detection_fps":2.3,"pid":229,"process_fps":5.0,"skipped_fps":0.0},"detection_fps":2.3,"detectors":{"coral":{"detection_start":1645452594.156627,"inference_speed":75.28,"pid":216}},"front":{"camera_fps":10.1,"capture_pid":231,"detection_fps":0.0,"pid":220,"process_fps":10.1,"skipped_fps":0.0},"garage":{"camera_fps":10.1,"capture_pid":232,"detection_fps":0.0,"pid":225,"process_fps":10.1,"skipped_fps":0.0},"gate":{"camera_fps":10.1,"capture_pid":235,"detection_fps":0.0,"pid":227,"process_fps":10.0,"skipped_fps":0.0},"service":{"storage":{"/dev/shm":{"free":64.3,"mount_type":"tmpfs","total":67.1,"used":2.8},"/media/frigate/clips":{"free":53895.1,"mount_type":"ext3","total":61285.2,"used":4263.3},"/media/frigate/recordings":{"free":53895.1,"mount_type":"ext3","total":61285.2,"used":4263.3},"/tmp/cache":{"free":1000.0,"mount_type":"tmpfs","total":1000.0,"used":0.0}},"uptime":1305,"version":"0.9.4-26ae608"}}

Operating system

Debian

Install method

Docker Compose

Coral version

Dev Board

Network connection

Wireless

Camera make and model

hikvision

Any other information that may be helpful

No response

blakeblackshear commented 2 years ago

I have no idea what to expect on that board, since his is the first I have seen someone use it. If the logs say TPU Found it is definitely being used. I know there are std and max speed drivers for some other Corals.

stale[bot] commented 2 years ago

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.