[Support]: Coral USB still having slow inference speed

[k2xl]

Describe the problem you are having

Finally got my hands on a coral USB.

Enabled 2 Reolink 810As. Getting inference speeds ~600.

Running on VirtualBox on my 10 year old old IBM Ideapad y560p.

I know VirtualBox isn't recommended (and neither is Reolink) with Frigate, but still thought I should be getting faster inference times.

Version

0.9.4-26AE608

Frigate config file

detectors:
  coral:
    type: edgetpu
    device: usb
cameras:
  front: # <------ Name the camera
    ffmpeg:
      inputs:
        - path: rtsp://admin:XXX@192.168.86.46:554/h264Preview_01_sub
          roles:
            - detect
            - rtmp
            - record
    record: # <----- Enable recording
      enabled: True
      retain_days: 0
      events:
        retain:
          default: 10
    detect:
      width: 640 # <---- update for your camera's resolution
      height: 360 # <---- update for your camera's resolution
      fps: 5
    motion:
      mask:
        - 467,10,468,36,154,34,160,9
  driveway: # <------ Name the camera
    ffmpeg:
      inputs:
        - path: rtsp://admin:XXX@192.168.86.46:554/h264Preview_03_sub
          roles:
            - detect
            - rtmp
            - record
    detect:
      width: 640 # <---- update for your camera's resolution
      height: 360 # <---- update for your camera's resolution
      fps: 5
    record: # <----- Enable recording
      enabled: True
      retain_days: 0
      events:
        retain:
          default: 10
    motion:
      mask:
      - 640,0,640,44,0,37,0,0

Relevant log output

n/a

FFprobe output from your camera

n/a.

Frigate stats

No response

Operating system

HassOS

Install method

HassOS Addon

Coral version

USB

Network connection

Wired

Camera make and model

Reolink 810B4-A https://www.amazon.com/gp/product/B07K734ZTW/ref=ppx_yo_dt_b_search_asin_title?ie=UTF8&psc=1

Any other information that may be helpful

No response

[bm55]

The frigate log does show that the coral is detected? "[2021-11-30 13:02:06] frigate.edgetpu INFO : TPU found"

[k2xl]

Yes

[blakeblackshear]

Unless there are other ways to pass the Coral to virtualbox, I think you are just experiencing the reason it's not a recommended setup. This just means the speed between the Coral and frigate is being throttled.

[k2xl]

Gonna get home assistant moved to run on host I guess. Thanks

[PrplHaz4]

you may want to try just moving frigate in case it's a hardware limitation

per coral spec

* Compatible with USB 2.0 but inferencing speed is slower.

[hugalafutro]

Definitely has to be in usb3 port and passes as usb3 too (not familiar with virtualbox to know to which extent that's possible) as that works for me in proxmox

I was getting inference speed in the hundreds when I passed the coral port through usb2 in proxmox with 1 fullhd rtmp stream at 10fps, then I passed it via usb3 port and now I'm getting 20-ish which is nice, but cpu was getting 50-70ish (12 core xeon E-2176M), I was hoping something around 10.

If I run the parrot recognition pycoral test on the same vm I get 16.5ms first (as it loads models) and the subsequent 4 are 4-5ms. which is the same I get running from the proxmox host directly so the vm passthru should not be issue as long as it's usb3.

[k2xl]

Just a quick update. I ended up buying a cheap mini pc for about 160$ (Cpu is an i5), installed ubuntu and home assistant on it (though side note i probably shouldve just done debian since ubuntu is not officially supported) and things are way faster. Was 50ms without coral and with coral 8.5ms.

[hugalafutro]

For what it's worth to anyone reading this in future, after I swapped from vm in proxmox to lxc in proxmox I got inference of 8ms.

[Normanras]

@hugalafutro I am reading this in the future! Or I guess the present.

Either way, facing this issue and I just moved my HA setup to Proxmox VM.

Can you share what you're running in the LXC? Docker in LXC with Frigate? Ubuntu LXC + Frigate?

[hugalafutro]

@Normanras Hi, I do no longer have the setup active as I moved frigate to Truenas Scale docker app, but when I did it was an Ubuntu lxc with docker inside. I'm not sure if it's still the case on 7.x proxmox, but when I was running the setup I had to add manually some lines inside the lxc's .conf file to facilitate the docker deamon access to gfx & cgroup and to passthru the usb. I also read that nowadays on latest proxmox it's enough to make privileged lxc and uninstall apparmor, but I haven't tried that.

I did not follow the whole tutorial from here, but you might find inspiration there: https://community.home-assistant.io/t/google-coral-usb-frigate-proxmox/383737 The lines I remember adding are mentioned in later part of it:

lxc.cgroup2.devices.allow: c 226:0 rwm
lxc.cgroup2.devices.allow: c 226:128 rwm
lxc.cgroup2.devices.allow: c 29:0 rwm
lxc.cgroup2.devices.allow: c 189:* rwm
lxc.apparmor.profile: unconfined
lxc.cgroup2.devices.allow: a
lxc.mount.entry: /dev/dri/renderD128 dev/dri/renderD128 none bind,optional,create=file 0, 0
lxc.mount.entry: /dev/bus/usb/002 dev/bus/usb/002 none bind,optional,create=dir 0, 0
lxc.cap.drop:
lxc.mount.auto: cgroup:rw

the 002 in lxc.mount.entry: /dev/bus/usb/002 dev/bus/usb/002 none bind,optional,create=dir 0, 0 has to physically correspond to the usb port you got coral connected to.

The setup was rock solid in lxc, but since I migrated from Truenas core to Truenas scale which natively supports docker it made more sense to run directly on storage. Hope any of it helps.

[Normanras]

This is incredibly helpful! Thank you @hugalafutro - I had this bookmarked that I was also going to try out later today. Appreciate your extra insight.

[ovizii]

For what it's worth to anyone reading this in future, after I swapped from vm in proxmox to lxc in proxmox I got inference of 8ms.

Are you passing through the USB Device in PVE or the whole USB Port?

[hugalafutro]

@ovizii not using the setup anymore, but i was passing thru the whole port