I can confirm frigate run on ESX 7.0 (ubuntu server 20.04) with google coral and ffmpeg hardware acceleration

[bezpodstawny]

I didn't know where to write but I can confirm that frigate works very well on ESX 7.0. I think it is worth adding this information to the readme file since there is information about Proxmox. My server configuration is based on the Xeon E3 1275v2 processor. Ubuntu works as one of the systems and frigate works on it in the docker. Google coral is connected not as a USB device but via the PCI Passtrough function together with the USB 3.1 gen2 controller. I used a PCIE controller. I disconnected the IGPU graphics card from the ESX server (not showing esx video) and made it available via Ubuntu PCI Passtrough. In the docker, just add the access options to the GPU device and it just works for 3 weeks. Advantages: trouble-free connection of Google Coral as if it were a physical computer and the operation of hardware acceleration of ffmpeg decoding.

---

---

{"asfalt":{"camera_fps":5.0,"capture_pid":37,"detection_fps":0.0,"frame_info":{"detect":1605478179.349492,"process":0.0},"pid":38,"process_fps":5.0,"skipped_fps":0.0},"brama":{"camera_fps":5.2,"capture_pid":54,"detection_fps":13.2,"frame_info":{"detect":1605478179.549442,"process":0.0},"pid":55,"process_fps":5.2,"skipped_fps":0.0},"budowa":{"camera_fps":5.0,"capture_pid":48,"detection_fps":0.0,"frame_info":{"detect":1605478179.465885,"process":0.0},"pid":50,"process_fps":5.0,"skipped_fps":0.0},"detection_fps":18.3,"detectors":{"coral":{"detection_start":0.0,"inference_speed":7.71,"pid":16}},"poludnie":{"camera_fps":5.1,"capture_pid":40,"detection_fps":0.0,"frame_info":{"detect":1605478179.444079,"process":0.0},"pid":44,"process_fps":5.1,"skipped_fps":0.0},"stodola":{"camera_fps":5.0,"capture_pid":58,"detection_fps":0.0,"frame_info":{"detect":1605478179.688275,"process":0.0},"pid":62,"process_fps":5.0,"skipped_fps":0.0},"taras":{"camera_fps":5.0,"capture_pid":52,"detection_fps":5.1,"frame_info":{"detect":1605478179.655248,"process":0.0},"pid":53,"process_fps":5.0,"skipped_fps":0.0}}

sudo docker run --rm --name frigate --privileged --shm-size=1024m --device /dev/dri:/dev/dri -v /dev/bus/usb:/dev/bus/usb -v /home/ws/frigate/config:/config:ro -v /etc/localtime:/etc/localtime:ro -p 5000:5000 -e FONTCONFIG_PATH='/etc/fonts' -e FRIGATE_RTSP_PASSWORD='password' blakeblackshear/frigate:0.7.3-amd64

https://tinkertry.com/vmware-vsphere-esxi-7-gpu-passthrough-ui-bug-workaround The guy from the link above was doing everything right but he had some strange problem, I suspect the reason was doing everything in one go. Please restart after each step in esx settings. So we can not turn off the esx graphics card without restart, set passtrouch to enable, connect gpu to the virtual machine and usb controller. This is a job for 4x reboot esx.

on esx paste: esxcli system settings kernel set -s vga -v FALSE is only way to passtrough GPU and on virtual machine set in parameters: svga.preset = FALSE

[blakeblackshear]

Great inference times too. Thanks for posting.

[Parlane]

Is your google coral a USB coral? You seem to be passing through the USB 3.1 hub not a PCI Coral like an M.2 Coral?

[bezpodstawny]

My google coral is a USB device. I actually paassing to virtual machine a USB 3.1 hub to which google coral is plugged. I did this to avoid problems with delays and the installation of google coral itself, which changes its id when turned on. Therefore, passing the id doesn't work to the virtual machine. USB 3 pcie hub does not cost much.

[alexsahka]

My google coral is a USB device. I actually paassing to virtual machine a USB 3.1 hub to which google coral is plugged. I did this to avoid problems with delays and the installation of google coral itself, which changes its id when turned on. Therefore, passing the id doesn't work to the virtual machine. USB 3 pcie hub does not cost much.

Do you think it will be an issue to use ESXI + M.2 to PCIe Card and pastry it to a virtual machine? I can't find Coral USB anywhere. Also, a question about your CPU what CPU do you have (model)? And how many cores do you dedicating to the virtual machine, what is the shared size of your RAM goes to the virtual machine?

[madmic1314]

My google coral is a USB device. I actually paassing to virtual machine a USB 3.1 hub to which google coral is plugged. I did this to avoid problems with delays and the installation of google coral itself, which changes its id when turned on. Therefore, passing the id doesn't work to the virtual machine. USB 3 pcie hub does not cost much.

Do you think it will be an issue to use ESXI + M.2 to PCIe Card and pastry it to a virtual machine? I can't find Coral USB anywhere. Also, a question about your CPU what CPU do you have (model)? And how many cores do you dedicating to the virtual machine, what is the shared size of your RAM goes to the virtual machine?

Is your USB adapater recognised for passthrough? If not you'll need one that does - they're cheap and not expensive.

[madmic1314]

@bezpodstawny When I boot Ubuntu server v20 with the vga passed through it powers off during boot. Remove the device and it's working - my adapter is an "Intel Corporation HD Graphics 530". The pass through works ok on Home Assistant but not Ubuntu?

Noticed running this under Ubuntu seems to use a lot less CPU - even without GPU acceleration.

[MEntOMANdo]

So just to confirm, @bezpodstawny, you've installed a PCIe physical card with USB 3.1 ports and then plugged the coral into that, and it does not change ID when you boot? But plugging the coral directly into your existing motheboard USB port does cause an ID change (thus preventing passthru).

Is M.2 stuff even eligible for PCI passthru? This M.2 stuff is confusing to me, I'm old enough to still have IDE cables laying around. :-) If I get the M.2 coral, I just want to be sure it can be passed thru without weird issues.

Edit: why shouldn't I just get the PCI-based Coral and plug it in and forget about USB?

[madmic1314]

@MEntOMANdo the internal USB3 ports were recognised by esx, but not available for pass through. I purchased one of these (£20 "StarTech.com 2 Port PCI Express USB 3.0 Controller Card w/ SATA Power - USB adapter - PCIe - USB 3.0 x 2 - PEXUSB3S23" ) which is on the HCL and works perfectly.

I added a PCI pass through device with the Coral device selected and that's all - think this is your edit comment, if so this is exactly what I've done.

Currently getting Inference Speed =< 8.1 - which I'm quite happy with.

I think my issue running this under HA was the id was changing sometimes which causes Frigate to crash out. Since hosting directly in Docker it's worked flawlessly (touch wood!) and also uses less CPU by a large amount (10W).

I'm not running pass through for the Intel adapter for HW accel, as Ubuntu crashes during boot with this enabled. As I'm drawling less power it's not a major to fix for now. One day...

[dvbit]

Thanks for this info, I was getting mad at having the Coral recognized in ESXI!

[dvbit]

my coral is visible in the vm doing the passthrough but is recognized as a Unichip VendorID. Passing it throuch the PCI adapter makes it visible with Google as VendorID?

Thanks

[dvbit]

I can confirm it is working. Installed a PCI USB card and passed it through and it finally works! be Thanks @bezpodstawny @blakeblackshear

[dakota]

For anybody looking at this wondering about the PCIe versions (Both the M.2 and the Mini-PCIe form factors), they do not currently working with ESXi (I don't know about other hypervisors). I've opened an issue https://github.com/google-coral/edgetpu/issues/343, but the response was that they do not officially support running in a VM.

My current thinking is to install the Coral into a small standalone PC that I have lying around, and then trying to get doods to work. Reason for frigate is that my VM server has a fairly decent GFX (4GB Nvidia GTX1650) installed which can be used for hwaccel decoding, but if that doesn't work well, then I think the PC I have might work with hwaccel (It's an older AM1 AMD Fusion based platform)

[t0ny-peng]

@dakota Thanks for the info. Do you expect the USB version of Coral to work with ESXi + USB passthrough? If so I can safely order one.

Update: I saw someone else saying it works. Still thx!

[mr-manuel]

@t0ny-peng you know maybe how he get it to work with USB passthrough?

[manalishi70]

I can confirm the following setup is working:

1. ESXi 6.0U3 on Dell T3610 XEON E5-1650v2

2. NVIDIA K600 Passthrough

3. UBUNTU 20.04 with Docker 20.10.12 and Nvidia-Docker 2.8.0

4. NVIDIA-SMI: Driver Version: 470.86 CUDA Version: 11.4 6 CAMS connected nvdia-smi on host

nvdia-smi in container

[lamtran0202]

I can confirm the following setup is working:

1. ESXi 6.0U3 on Dell T3610 XEON E5-1650v2
2. NVIDIA K600 Passthrough
3. UBUNTU 20.04 with Docker 20.10.12 and Nvidia-Docker 2.8.0
4. NVIDIA-SMI: Driver Version: 470.86 CUDA Version: 11.4 6 CAMS connected nvdia-smi on host

nvdia-smi in container

can you help? how to add quadro card in host! i have quadro T400 . but can not see into hardware of host

[heapxor]

hello, can anyone advise if PCIe to USB card is required or is it enough to add usb device, without passthrough? Can i somehow check that Coral usb is visible/working within HAS? Thanks

[mr-manuel]

Hello @heapxor, at the moment a PCI to USB adapter is required, since the Google Coral changes ID after the first time you use it. I spent a few days to find a solution for direct passthrough, but I found nothing that worked reliably.

See also this comments: https://github.com/blakeblackshear/frigate/issues/305#issuecomment-731185496 https://github.com/blakeblackshear/frigate/issues/305#issuecomment-800678478 https://github.com/blakeblackshear/frigate/issues/2178#issuecomment-962614526

Maybe some day there is a Google Coral driver for ESX/vSphere, if there are enough requests…

[heapxor]

hi, is it possible to say if that one is compatible with linux kernel 5x ? https://www.amazon.co.uk/gp/product/B009AT5SVS/

i tried 3 different brands no luck ; only usb 2.x or not compatible at all.

[heapxor]

hi, @bezpodstawny i dont get it .. how did u enable hw acceleration for ffmpeg? thanks

[justushowey]

Nice catch on the svga present option in the VM. Is there any way to see your Frigate config?

[dailow]

@MEntOMANdo the internal USB3 ports were recognised by esx, but not available for pass through. I purchased one of these (£20 "StarTech.com 2 Port PCI Express USB 3.0 Controller Card w/ SATA Power - USB adapter - PCIe - USB 3.0 x 2 - PEXUSB3S23" ) which is on the HCL and works perfectly.

I added a PCI pass through device with the Coral device selected and that's all - think this is your edit comment, if so this is exactly what I've done.

Currently getting Inference Speed =< 8.1 - which I'm quite happy with.

I think my issue running this under HA was the id was changing sometimes which causes Frigate to crash out. Since hosting directly in Docker it's worked flawlessly (touch wood!) and also uses less CPU by a large amount (10W).

I'm not running pass through for the Intel adapter for HW accel, as Ubuntu crashes during boot with this enabled. As I'm drawling less power it's not a major to fix for now. One day...

Did you have to add anything to the VMX file for this to work? For some reason I get USB errors when using this exact same card after it runs for a bit. I verified I don't see the same issue on my Mac, so I know it is not the cable/coral itself. Maybe the PCI-E card is bad, but it only happens when I use a USB 3.0 cable. If I use a USB 2.0 cable and force it to run at lower speeds it works fine.

[sanderlv]

BUMP - I have an intel NUC running latest ESXi 8.0x. I also cant get coral to pass through with the google ID.

Is there any one having had luck in that?I cant add anoter PCI device.

[sanderlv]

Some progress for ESXi... but still no real luck

https://williamlam.com/2023/05/google-coral-usb-edge-tpu-accelerator-on-esxi.html

[mr-manuel]

Some progress for ESXi... but still no real luck

https://williamlam.com/2023/05/google-coral-usb-edge-tpu-accelerator-on-esxi.html

Nice, thanks! For me it's working now :-D

Host Intel NUC10 ESXi 7.0 U3 USB Coral

VM Debian 10

Here the difference between before and after. WOW!

[SpeedBlack]

Some progress for ESXi... but still no real luck https://williamlam.com/2023/05/google-coral-usb-edge-tpu-accelerator-on-esxi.html

Nice, thanks! For me it's working now :-D

Host Intel NUC10 ESXi 7.0 U3 USB Coral

VM Debian 10

Here the difference between before and after. WOW!

Hello @mr-manuel , Can you confirm that you use this method without PCI PASSTHROUGH ? I can't get the PCI to work on my PCI PASSTHROUGH (Frozen VM). Thanks !

[mr-manuel]

I did exacly what is described in https://williamlam.com/2023/05/google-coral-usb-edge-tpu-accelerator-on-esxi.html. No other steps. No PCI passthrough.

Update: I never got it to run stable for longer than 6 hours. https://github.com/blakeblackshear/frigate/issues/7427

[goldserve]

I followed the instructions and while I got the USB TPU to be recognized by the VM running OMV (Debian) and docker, the entire VM would lock up and it turned out to the many CPU locks on strange USB errors. Once I unplugged the coral device, ESXI, Vcenter would behave much better and I could restart the entire VM. I've tried supplying external power, different USB ports and while it might work initially, USB errors seem to kill the entire VM. I'm going to install an M.2 USB PCIe card and pass that through for Coral to see if stability is better.

[theawes0megamer]

Getting that USB PCIe controller and putting it into my rack mounted server fixed my issues where the coral would crash after visiting the webpage for frigate after a couple minutes, causing frigate to crash. Took me two weeks to figure out because this issue was removed from the docs.