search

blakeblackshear / frigate

NVR with realtime local object detection for IP cameras
https://frigate.video
MIT License
18.13k stars 1.65k forks source link

[Detector Support]: m.2 B+E coral detected but not used #9311

Closed luciandf closed 7 months ago

luciandf commented 8 months ago

Describe the problem you are having

I am running frigate 0.12.1 in a privileged proxmox lxc container where I am running docker with portainer.

At first I had a mini pcie coral which worked fine with this setup, until due to a power cut it died. I then bought an M.2 B+E coral. I am using the same container but for some reason which I cannot understand the detector does not appear to be used.

The status now is as follows:

2024-01-12 01:20:01.298990027  [INFO] Starting Frigate...
2024-01-12 01:20:02.297846170  [2024-01-12 01:20:02] frigate.app                    INFO    : Starting Frigate (0.12.1-367d724)
2024-01-12 01:20:02.328719099  [2024-01-12 01:20:02] peewee_migrate                 INFO    : Starting migrations
2024-01-12 01:20:02.332544777  [2024-01-12 01:20:02] peewee_migrate                 INFO    : There is nothing to migrate
2024-01-12 01:20:02.347392763  [2024-01-12 01:20:02] detector.coral                 INFO    : Starting detection process: 355
2024-01-12 01:20:02.349322295  [2024-01-12 01:20:02] frigate.app                    INFO    : Output process started: 357
2024-01-12 01:20:02.355837834  [2024-01-12 01:20:02] frigate.app                    INFO    : Camera processor started for gate_entrance: 362
2024-01-12 01:20:02.359939213  [2024-01-12 01:20:02] frigate.app                    INFO    : Camera processor started for veranda_entrance: 363
2024-01-12 01:20:02.364345330  [2024-01-12 01:20:02] frigate.app                    INFO    : Camera processor started for terasa_back: 365
2024-01-12 01:20:02.368645644  [2024-01-12 01:20:02] frigate.app                    INFO    : Capture process started for gate_entrance: 366
2024-01-12 01:20:02.372562274  [2024-01-12 01:20:02] frigate.app                    INFO    : Capture process started for veranda_entrance: 367
2024-01-12 01:20:02.379213270  [2024-01-12 01:20:02] frigate.app                    INFO    : Capture process started for terasa_back: 372
2024-01-12 01:20:02.383559863  [2024-01-12 01:20:02] frigate.detectors.plugins.edgetpu_tfl INFO    : Attempting to load TPU as pci:0
2024-01-12 01:20:02.388019652  [2024-01-12 01:20:02] frigate.detectors.plugins.edgetpu_tfl INFO    : TPU found

However, I will be standing in front of the camera and I get no detection even. When I switch to a CPU detection, it works fine.

The lxc container conf file is:

arch: amd64
cores: 4
features: nesting=1
hostname: frigate
memory: 8192
mp0: /mnt/recording,mp=/opt/frigate/storage
net0: name=eth0,bridge=vmbr0,firewall=1,gw=192.168.1.1,hwaddr=BA:6F:A6:4B:43:6F,ip=192.168.1.66/24,type=veth
onboot: 1
ostype: debian
rootfs: local:1000/vm-1000-disk-0.raw,size=32G
swap: 8192
lxc.cgroup2.devices.allow: c 120:0 rwm
lxc.mount.entry: /dev/apex_0 dev/apex_0 none bind,optional,create=file
lxc.cgroup2.devices.allow: c 226:0 rwm
lxc.cgroup2.devices.allow: c 226:128 rwm
lxc.mount.entry: /dev/dri/renderD128 dev/dri/renderD128 none bind,optional,create=file 0, 0
lxc.mount.entry: /dev/dri/card0 dev/dri/card0 none bind,optional,create=file 0, 0
lxc.mount.entry: /dev/dri dev/dri none bind,optional,create=dir

Version

0.12.1

Frigate config file

version: "3.9"
services:
  frigate:
    container_name: frigate
    group_add:
      - '44'                                      # add video group
      - '105'                                     # add render group
    privileged: true # this may not be necessary for all setups
    restart: unless-stopped
    image: ghcr.io/blakeblackshear/frigate:stable
    cap_add:
      - CAP_PERFMON
    shm_size: "2048mb" # update for your cameras based on calculation above
    devices:
      - /dev/apex_0:/dev/apex_0 # passes a PCIe Coral, follow driver instructions here https://coral.ai/docs/m2/get-started/#2a-on-linux
      - /dev/dri/card0:/dev/dri/card0
      - /dev/dri/renderD128:/dev/dri/renderD128 # for intel hwaccel, needs to be updated for your hardware
    volumes:
      - /etc/localtime:/etc/localtime:ro
      - /opt/frigate/config.yml:/config/config.yml
#      - /opt/frigate/:/config/
      - /opt/frigate/storage:/media/frigate
      - /db:/db
      - type: tmpfs # Optional: 1GB of memory, reduces SSD/SD Card wear
        target: /tmp/cache
        tmpfs:
          size: 6000000000
    ports:
      - "5000:5000"
      - "8554:8554" # RTSP feeds
      - "8555:8555/tcp" # WebRTC over tcp
      - "8555:8555/udp" # WebRTC over udp
    environment:
      FRIGATE_RTSP_PASSWORD: "super secred password"

docker-compose file or Docker CLI command

version: "3.9"
services:

  frigate:
    container_name: frigate
    group_add:
      - '44'                                      # add video group
      - '105'                                     # add render group
    privileged: true # this may not be necessary for all setups
    restart: unless-stopped
    image: ghcr.io/blakeblackshear/frigate:stable
#    image: ghcr.io/blakeblackshear/frigate:0.13.0-beta4
    cap_add:
      - CAP_PERFMON
    shm_size: "2048mb" # update for your cameras based on calculation above
    devices:
      - /dev/apex_0:/dev/apex_0 # passes a PCIe Coral, follow driver instructions here https://coral.ai/docs/m2/get-started/#2a-on-linux
      - /dev/dri/card0:/dev/dri/card0
      - /dev/dri/renderD128:/dev/dri/renderD128 # for intel hwaccel, needs to be updated for your hardware
    volumes:
      - /etc/localtime:/etc/localtime:ro
      - /opt/frigate/config.yml:/config/config.yml
#      - /opt/frigate/:/config/
      - /opt/frigate/storage:/media/frigate
      - /db:/db
      - type: tmpfs # Optional: 1GB of memory, reduces SSD/SD Card wear
        target: /tmp/cache
        tmpfs:
          size: 6000000000
    ports:
      - "5000:5000"
      - "8554:8554" # RTSP feeds
      - "8555:8555/tcp" # WebRTC over tcp
      - "8555:8555/udp" # WebRTC over udp
    environment:
      FRIGATE_RTSP_PASSWORD: "frig@ODIN2023!"

Relevant log output

2024-01-12 01:20:01.298990027  [INFO] Starting Frigate...
2024-01-12 01:20:02.297846170  [2024-01-12 01:20:02] frigate.app                    INFO    : Starting Frigate (0.12.1-367d724)
2024-01-12 01:20:02.328719099  [2024-01-12 01:20:02] peewee_migrate                 INFO    : Starting migrations
2024-01-12 01:20:02.332544777  [2024-01-12 01:20:02] peewee_migrate                 INFO    : There is nothing to migrate
2024-01-12 01:20:02.347392763  [2024-01-12 01:20:02] detector.coral                 INFO    : Starting detection process: 355
2024-01-12 01:20:02.349322295  [2024-01-12 01:20:02] frigate.app                    INFO    : Output process started: 357
2024-01-12 01:20:02.355837834  [2024-01-12 01:20:02] frigate.app                    INFO    : Camera processor started for gate_entrance: 362
2024-01-12 01:20:02.359939213  [2024-01-12 01:20:02] frigate.app                    INFO    : Camera processor started for veranda_entrance: 363
2024-01-12 01:20:02.364345330  [2024-01-12 01:20:02] frigate.app                    INFO    : Camera processor started for terasa_back: 365
2024-01-12 01:20:02.368645644  [2024-01-12 01:20:02] frigate.app                    INFO    : Capture process started for gate_entrance: 366
2024-01-12 01:20:02.372562274  [2024-01-12 01:20:02] frigate.app                    INFO    : Capture process started for veranda_entrance: 367
2024-01-12 01:20:02.379213270  [2024-01-12 01:20:02] frigate.app                    INFO    : Capture process started for terasa_back: 372
2024-01-12 01:20:02.383559863  [2024-01-12 01:20:02] frigate.detectors.plugins.edgetpu_tfl INFO    : Attempting to load TPU as pci:0
2024-01-12 01:20:02.388019652  [2024-01-12 01:20:02] frigate.detectors.plugins.edgetpu_tfl INFO    : TPU found

Operating system

Proxmox

Install method

Docker Compose

Coral version

M.2

Any other information that may be helpful

No response

NickM-27 commented 8 months ago

What is the inference speed on the system page with it?

luciandf commented 8 months ago

What is the inference speed on the system page with it?

it shows 6.64 ms

NickM-27 commented 8 months ago

Then that means it is being used. Inference speed will only be set if it gets a successful inference back

luciandf commented 8 months ago

i see! thank you! i got detected by one camera just now. maybe this is a problem because they are ip cameras bullet imou 4mp

luciandf commented 8 months ago

sorry to start again but I have one more question: is there a reason why one camera will detect a person better than the others? All are identical and are placed in places with good wifi signal, yet they will not detect all the images where people are in them.

Is there a way to debug this? Sadly I am not at home anymore and can only remotely look at logs and everything but I cannot test the cameras myself (unless an intruder is willing to participate in a quality control testing...).

NickM-27 commented 8 months ago

The default model is not trained on security camera images as it is a demo model from Google trained on the COCO dataset. This means it can work very differently from camera to camera. Frigate does have a debug live view where you can see what is detected on the camera and get more info.

luciandf commented 8 months ago

The default model is not trained on security camera images as it is a demo model from Google trained on the COCO dataset. This means it can work very differently from camera to camera. Frigate does have a debug live view where you can see what is detected on the camera and get more info.

thank you for this! I have no experience with training models and all of this, but is there a way to load a better model for this coral? it just seems quite random. One camera detects almost all people in an image, which other (identical) cameras will just detect people in one or two frames. Almost like the rest are ignored and not processed.

NickM-27 commented 8 months ago

A recording of debug view would be helpful to see so we can understand what exactly is going on (with motion and bounding boxes enabled)

There are different options for better models, Frigate+ is one of those options https://github.com/blakeblackshear/frigate/discussions/7932

github-actions[bot] commented 7 months ago

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.