search

blakeblackshear / frigate

NVR with realtime local object detection for IP cameras
https://frigate.video
MIT License
18.93k stars 1.73k forks source link

[Detector Support]: Coral interference speed 20-30ms #9428

Closed ViezeVisNL closed 9 months ago

ViezeVisNL commented 9 months ago

Describe the problem you are having

Hey,

I've just setup a Google Coral USB but I noticed the inference speed (~20-30ms is just a little lower than via openvino (~40ms). Is there something I forgot? I'd expected at least 10ms or something.

Based on the config I just setup the coral detector. No width/heights etc. https://docs.frigate.video/configuration/detectors/#single-usb-coral

detectors:
  coral:
    type: edgetpu
    device: usb

Version

0.12.1-367D724

Frigate config file

#AI detectors
detectors:
  coral:
    type: edgetpu
    device: usb

#  ov:
#    type: openvino
#    device: AUTO
#    model:
#      path: /openvino-model/ssdlite_mobilenet_v2.xml

# hardware acceleration
#ffmpeg:
#  hwaccel_args: -hwaccel vaapi -hwaccel_device /dev/dri/renderD128

#model:
#  width: 300
#  height: 300
#  input_tensor: nhwc
#  input_pixel_format: bgr
#  labelmap_path: /openvino-model/coco_91cl_bkgr.txt

#go2rtc
go2rtc:
#  log:
#      exec: trace
  streams:
    reolink:
      - "ffmpeg:http://192.168.2.65/flv?port=1935&app=bcs&stream=channel0_main.bcs&user=admin&password=****#video=copy#audio=copy#audio=opus"
      - rtsp://admin:****@192.168.2.65:554/h264Preview_01_sub
      #- rtsp://admin:****@192.168.2.65:554/h264Preview_01_main
      - "ffmpeg:reolink#audio=opus"
    reolink_sub:
      - "ffmpeg:http://192.168.2.65/flv?port=1935&app=bcs&stream=channel0_ext.bcs&user=admin&password=****"
  #ffmpeg:
    #bin: ffmpeg
    #volume: "-af \"volume=30dB\""

docker-compose file or Docker CLI command

/

Relevant log output

2024-01-24 09:00:30.812581992  [INFO] Starting Frigate...
2024-01-24 09:00:32.082510784  [2024-01-24 10:00:32] frigate.app                    INFO    : Starting Frigate (0.12.1-367d724)
2024-01-24 09:00:32.104391573  [2024-01-24 10:00:32] frigate.app                    INFO    : Creating directory: /tmp/cache
2024-01-24 09:00:32.104974914  [2024-01-24 10:00:32] peewee_migrate                 INFO    : Starting migrations
2024-01-24 09:00:32.110633878  [2024-01-24 10:00:32] peewee_migrate                 INFO    : There is nothing to migrate
2024-01-24 09:00:32.155114927  [2024-01-24 10:00:32] frigate.app                    INFO    : Output process started: 228
2024-01-24 09:00:32.179631787  [2024-01-24 10:00:32] frigate.app                    INFO    : Camera processor started for reolink: 234
2024-01-24 09:00:32.189172989  [2024-01-24 10:00:32] frigate.app                    INFO    : Capture process started for reolink: 236
2024-01-24 09:00:34.958055655  [2024-01-24 10:00:32] detector.coral                 INFO    : Starting detection process: 226
2024-01-24 09:00:34.958059196  [2024-01-24 10:00:32] frigate.detectors.plugins.edgetpu_tfl INFO    : Attempting to load TPU as usb
2024-01-24 09:00:34.963325714  [2024-01-24 10:00:34] frigate.detectors.plugins.edgetpu_tfl INFO    : TPU found

Operating system

HassOS

Install method

HassOS Addon

Coral version

USB

Any other information that may be helpful

No response

leccelecce commented 9 months ago

It will help to know the physical hardware you are running on (i.e. Pi4, mini PC, Dell R620 rack server), whether you are running bare-metal / container / virtual machine, what the wider resource usage on the machine is.

ViezeVisNL commented 9 months ago

Ah. Ofcourse.

Running on HP ProDesk 600 G3 Mini with Proxmox. HAOS is running in a Proxmox VM. cpu: i5-6500t ram: 8GB ddr4 storage: 512gb m2 ssd 12% CPU usage (2 cores) / 69% Mem Usage (4GiB)

leccelecce commented 9 months ago

What jumps out there is that you're a) running in a VM b) running a fairly old CPU.

I would take a look at https://github.com/blakeblackshear/frigate/discussions/1837 but warn that it might just confuse you further given the mish-mash of opinions. A lot of the discussion is about PCIe M2 corals (rather than USB) or about passing through graphics renderers (not what you're trying to do) so you need to focus on the USB-to-VM bits. In short though, many (though not all) Coral USB users report significant improvements moving out of VM into bare metal/LXC container. I went from 18ms to 8ms on a Dell R220 previously.

The other thing you can look at is how you are passing through the Coral USB device from Proxmox to your VM - depending on how this is being done, Qemu (the Proxmox VM layer) may be emulating the USB port on the CPU which hugely impacts bandwidth. You may need to pass the whole USB PCI device through to the VM.

Finally, if you have Proxmox you're probably looking to run other services on it. Personally, I ended up moving Frigate out of Proxmox, because some of my various other services were latency- and uptime-sensitive, which isn't ideal alongside an application that sucks CPU/GPU 24/7. One to consider if you have the power/financial budget for a standalone device, particularly if your number of cameras/resolution will grow...

ViezeVisNL commented 9 months ago

What jumps out there is that you're a) running in a VM b) running a fairly old CPU.

I would take a look at #1837 but warn that it might just confuse you further given the mish-mash of opinions. A lot of the discussion is about PCIe M2 corals (rather than USB) or about passing through graphics renderers (not what you're trying to do) so you need to focus on the USB-to-VM bits. In short though, many (though not all) Coral USB users report significant improvements moving out of VM into bare metal/LXC container. I went from 18ms to 8ms on a Dell R220 previously.

The other thing you can look at is how you are passing through the Coral USB device from Proxmox to your VM - depending on how this is being done, Qemu (the Proxmox VM layer) may be emulating the USB port on the CPU which hugely impacts bandwidth. You may need to pass the whole USB PCI device through to the VM.

Finally, if you have Proxmox you're probably looking to run other services on it. Personally, I ended up moving Frigate out of Proxmox, because some of my various other services were latency- and uptime-sensitive, which isn't ideal alongside an application that sucks CPU/GPU 24/7. One to consider if you have the power/financial budget for a standalone device, particularly if your number of cameras/resolution will grow...

I've passed through the PCI "200 series/Z370 Chipset Family USB 3.0 xHCI Controller" and it's giving me a Inference Speed of 10.06ms right now, which should be way more acceptable now with the Coral device 🎉 CPU currently running at ~ 10% and memory 57%. But this might be due to the restart of the VM.

I'll give it a go and see now :)

ViezeVisNL commented 9 months ago

9.7ms. Cant complain. Case closed. This USB passthrough via Proxmox was a definite fix. Thanks!