Closed ViezeVisNL closed 9 months ago
It will help to know the physical hardware you are running on (i.e. Pi4, mini PC, Dell R620 rack server), whether you are running bare-metal / container / virtual machine, what the wider resource usage on the machine is.
Ah. Ofcourse.
Running on HP ProDesk 600 G3 Mini with Proxmox. HAOS is running in a Proxmox VM. cpu: i5-6500t ram: 8GB ddr4 storage: 512gb m2 ssd 12% CPU usage (2 cores) / 69% Mem Usage (4GiB)
What jumps out there is that you're a) running in a VM b) running a fairly old CPU.
I would take a look at https://github.com/blakeblackshear/frigate/discussions/1837 but warn that it might just confuse you further given the mish-mash of opinions. A lot of the discussion is about PCIe M2 corals (rather than USB) or about passing through graphics renderers (not what you're trying to do) so you need to focus on the USB-to-VM bits. In short though, many (though not all) Coral USB users report significant improvements moving out of VM into bare metal/LXC container. I went from 18ms to 8ms on a Dell R220 previously.
The other thing you can look at is how you are passing through the Coral USB device from Proxmox to your VM - depending on how this is being done, Qemu (the Proxmox VM layer) may be emulating the USB port on the CPU which hugely impacts bandwidth. You may need to pass the whole USB PCI device through to the VM.
Finally, if you have Proxmox you're probably looking to run other services on it. Personally, I ended up moving Frigate out of Proxmox, because some of my various other services were latency- and uptime-sensitive, which isn't ideal alongside an application that sucks CPU/GPU 24/7. One to consider if you have the power/financial budget for a standalone device, particularly if your number of cameras/resolution will grow...
What jumps out there is that you're a) running in a VM b) running a fairly old CPU.
I would take a look at #1837 but warn that it might just confuse you further given the mish-mash of opinions. A lot of the discussion is about PCIe M2 corals (rather than USB) or about passing through graphics renderers (not what you're trying to do) so you need to focus on the USB-to-VM bits. In short though, many (though not all) Coral USB users report significant improvements moving out of VM into bare metal/LXC container. I went from 18ms to 8ms on a Dell R220 previously.
The other thing you can look at is how you are passing through the Coral USB device from Proxmox to your VM - depending on how this is being done, Qemu (the Proxmox VM layer) may be emulating the USB port on the CPU which hugely impacts bandwidth. You may need to pass the whole USB PCI device through to the VM.
Finally, if you have Proxmox you're probably looking to run other services on it. Personally, I ended up moving Frigate out of Proxmox, because some of my various other services were latency- and uptime-sensitive, which isn't ideal alongside an application that sucks CPU/GPU 24/7. One to consider if you have the power/financial budget for a standalone device, particularly if your number of cameras/resolution will grow...
I've passed through the PCI "200 series/Z370 Chipset Family USB 3.0 xHCI Controller" and it's giving me a Inference Speed of 10.06ms right now, which should be way more acceptable now with the Coral device 🎉 CPU currently running at ~ 10% and memory 57%. But this might be due to the restart of the VM.
I'll give it a go and see now :)
9.7ms. Cant complain. Case closed. This USB passthrough via Proxmox was a definite fix. Thanks!
Describe the problem you are having
Hey,
I've just setup a Google Coral USB but I noticed the inference speed (~20-30ms is just a little lower than via openvino (~40ms). Is there something I forgot? I'd expected at least 10ms or something.
Based on the config I just setup the coral detector. No width/heights etc. https://docs.frigate.video/configuration/detectors/#single-usb-coral
Version
0.12.1-367D724
Frigate config file
docker-compose file or Docker CLI command
Relevant log output
Operating system
HassOS
Install method
HassOS Addon
Coral version
USB
Any other information that may be helpful
No response