search

blakeblackshear / frigate

NVR with realtime local object detection for IP cameras
https://frigate.video
MIT License
19.23k stars 1.76k forks source link

[EdgeTPU Support]: #4073

Closed Dutch317 closed 1 year ago

Dutch317 commented 2 years ago

Describe the problem you are having

Frigate shuts down after a few minutes with a error of: F driver/usb/usb_driver.cc:1148] HandleQueuedBulkIn transfer in failed. Unknown: USB transfer error 1 [LibUsbDataInCallback] Fatal Python error: Aborted This only started after moving to a Coral USB TPU.

Version

0.11.1-2EADA21

Frigate config file

mqtt:
  host: 192.168.2.20
  port: 1883
  topic_prefix: frigate
  client_id: frigate
  user: ***************
  password: ************
  # state_interval: 60
rtmp:
  # Optional: Enable the RTMP stream (default: True)
  enabled: False   
detectors:
  coral:
    type: edgetpu
    device: usb
cameras:
  Driveway1:
    ffmpeg:
      inputs:
        - path: rtsp://***********@192.168.2.174:554//h264Preview_01_main
          roles:
            - record
        - path: rtsp://*************@192.168.2.174:554//h264Preview_01_sub
          roles:
            - detect
      output_args:
        record: -f segment -segment_time 10 -segment_format mp4 -reset_timestamps 1 -strftime 1 -c:v copy -c:a aac
    detect:
      width: 640
      height: 480
      fps: 7
    record:
      enabled: True
      retain:
        days: 0
        mode: active_objects
      events:
        pre_capture: 5
        post_capture: 5
        retain:
          default: 2
        required_zones:
          - basement_door
          - back_door
    snapshots:
      enabled: true
      timestamp: false
      bounding_box: true
      retain:
        default: 2
    objects:
      track:
        - person
        - dog
        - cat
      filters:
        person:
          threshold: 0.7
    motion:
      mask:
        - 403,480,618,480,618,438,401,443
        - 328,277,70,228,51,59,157,44,160,91,268,102,349,91,352,0,0,0,0,480,148,480
        - 426,75,427,0,640,0,640,54,640,480,519,96
    zones:
      basement_door:
        coordinates: 67,282,291,274,301,92,85,74
        objects:
          - person
          - dog
          - cat
      back_door:
        coordinates: 67,282,291,274,301,92,85,74
        objects:
          - person
          - dog
          - cat
  Driveway2:
    ffmpeg:
      inputs:
        - path: rtsp://*********@192.168.2.185:554//h264Preview_01_main
          roles:
            - record
        - path: rtsp://**********@192.168.2.185:554//h264Preview_01_sub
          roles:
            - detect
      output_args:
        record: -f segment -segment_time 10 -segment_format mp4 -reset_timestamps 1 -strftime 1 -c:v copy -c:a aac
    detect:
      width: 640
      height: 480
      fps: 7
    record:
      enabled: True
      retain:
        days: 0
        mode: active_objects
      events:
        pre_capture: 5
        post_capture: 5
        retain:
          default: 2
        required_zones:
          - basement_door
          - driveway
          - grass
    snapshots:
      enabled: true
      timestamp: false
      bounding_box: true
      retain:
        default: 2
    objects:
      track:
        - person
        - dog
        - cat
      filters:
        person:
          threshold: 0.7
    motion:
      mask:
        - 403,480,618,480,618,438,401,443
        - 0,0,0,67,496,56,640,0
    zones:
      basement_door:
        coordinates: 640,0,640,227,523,276,523,0
        objects:
          - person
          - dog
          - cat
      driveway:
        coordinates: 154,480,187,69,303,72,640,343,640,480
        objects:
          - person
          - dog
          - cat
      grass:
        coordinates: 0,480,0,70,191,66,164,480
        objects:
          - person
          - dog
          - cat
  FrontPorch1:
    ffmpeg:
      inputs:
        - path: rtsp://**************@192.168.2.179:554//h264Preview_01_main
          roles:
            - record
        - path: rtsp://*************@192.168.2.179:554//h264Preview_01_sub
          roles:
            - detect
      output_args:
        record: -f segment -segment_time 10 -segment_format mp4 -reset_timestamps 1 -strftime 1 -c:v copy -c:a aac
    detect:
      width: 640
      height: 480
      fps: 7
    record:
      enabled: True
      retain:
        days: 0
        mode: active_objects
      events:
        pre_capture: 5
        post_capture: 5
        retain:
          default: 2
        required_zones:
          - porch
          - sidewalk
          - road
    snapshots:
      enabled: true
      timestamp: false
      bounding_box: true
      retain:
        default: 2
    objects:
      track:
        - person
        - dog
        - cat
      filters:
        person:
          threshold: 0.7
    motion:
      mask:
        - 403,480,618,480,618,438,401,443
        - 73,59,130,58,134,120,79,128
        - 452,85,450,54,585,54,576,107
        - 205,120,259,119,263,75,201,73
        - 334,293,334,242,298,242,298,296
        - 318,336,351,335,352,302,317,302
    zones:
      sidewalk:
        coordinates: 640,267,197,77,172,135,640,410
        objects:
          - person
          - dog
          - cat
      road:
        coordinates: 271,122,273,60,640,149,640,256
        objects:
          - person
          - dog
          - cat
      porch:
        coordinates: 394,247,266,277,462,480,112,480,70,134,179,120,275,227,358,220
        objects:
          - person
          - dog
          - cat
  FrontPorch2:
    ffmpeg:
      inputs:
        - path: rtsp://************@192.168.2.216:554//h264Preview_01_main
          roles:
            - record
        - path: rtsp://**************@192.168.2.216:554//h264Preview_01_sub
          roles:
            - detect
      output_args:
        record: -f segment -segment_time 10 -segment_format mp4 -reset_timestamps 1 -strftime 1 -c:v copy -c:a aac
    detect:
      width: 640
      height: 480
      fps: 7
    record:
      enabled: True
      retain:
        days: 0
        mode: active_objects
      events:
        pre_capture: 5
        post_capture: 5
        retain:
          default: 2
        required_zones:
          - front_sidewalk
          - front_door_face
    objects:
      track:
        - person
        - dog
        - cat
      filters:
        person:
          threshold: 0.7 
    snapshots:
      enabled: true
      timestamp: false
      bounding_box: true
      retain:
        default: 2
    motion:
      mask:
        - 640,436,399,439,401,480,640,480
        - 0,0,510,0,640,0,640,0,640,53,386,70,0,216
        - 640,346,640,25,568,26,377,143
    zones:
      front_door_face:
        coordinates: 271,183,443,429,640,480,0,480,0,299
        objects:
          - person
          - dog
          - cat
      front_sidewalk:
        coordinates: 0,201,309,98,519,53,560,85,0,293
        objects:
          - person
          - dog
          - cat
  Backyard:
    ffmpeg:
      inputs:
        - path: rtsp://**********@192.168.2.187:554//h264Preview_01_main
          roles:
            - record
        - path: rtsp://**************@192.168.2.187:554//h264Preview_01_sub
          roles:
            - detect
      output_args:
        record: -f segment -segment_time 10 -segment_format mp4 -reset_timestamps 1 -strftime 1 -c:v copy -c:a aac
    detect:
      width: 640
      height: 480
      fps: 7
    record:
      enabled: True
      retain:
        days: 0
        mode: active_objects
      events:
        pre_capture: 5
        post_capture: 5
        retain:
          default: 2
        required_zones:
          - back_yard
    objects:
      track:
        - person
        - dog
        - cat
        - horse
        - sheep
        - cow
        - bear
      filters:
        person:
          threshold: 0.7 
    snapshots:
      enabled: true
      timestamp: false
      bounding_box: true
      retain:
        default: 2
    motion:
      mask:
        - 640,436,399,439,401,480,640,480
        - 267,161,0,200,0,0,640,0,640,103,595,101,519,131,408,149,361,153
    zones:
      back_yard:
        coordinates: 252,168,416,135,573,108,640,123,640,480,0,480,0,194
        objects:
          - person
          - dog
          - cat
          - horse
          - sheep
          - cow
          - bear

docker-compose file or Docker CLI command

-

Relevant log output

[s6-init] making user provided files available at /var/run/s6/etc...exited 0.
[s6-init] ensuring user provided files have correct perms...exited 0.
[fix-attrs.d] applying ownership & permissions fixes...
[fix-attrs.d] done.
[cont-init.d] executing container initialization scripts...
[cont-init.d] done.
[services.d] starting services
[services.d] done.
[2022-10-12 18:04:47] frigate.app                    INFO    : Starting Frigate (0.11.1-2eada21)
[2022-10-12 18:04:47] frigate.app                    INFO    : Creating directory: /tmp/cache
Starting migrations
[2022-10-12 18:04:47] peewee_migrate                 INFO    : Starting migrations
There is nothing to migrate
[2022-10-12 18:04:47] peewee_migrate                 INFO    : There is nothing to migrate
[2022-10-12 18:04:47] frigate.app                    INFO    : Output process started: 233
[2022-10-12 18:04:47] detector.coral                 INFO    : Starting detection process: 232
[2022-10-12 18:04:47] ws4py                          INFO    : Using epoll
[2022-10-12 18:04:47] frigate.mqtt                   INFO    : Turning off recordings for Driveway2 via mqtt
[2022-10-12 18:04:47] frigate.edgetpu                INFO    : Attempting to load TPU as usb
[2022-10-12 18:04:47] frigate.mqtt                   INFO    : Turning off snapshots for Driveway2 via mqtt
[2022-10-12 18:04:51] frigate.edgetpu                INFO    : TPU found
[2022-10-12 18:04:47] frigate.mqtt                   INFO    : Turning off detection for Driveway2 via mqtt
[2022-10-12 18:04:47] frigate.app                    INFO    : Camera processor started for Driveway1: 241
[2022-10-12 18:04:47] frigate.app                    INFO    : Camera processor started for Driveway2: 243
[2022-10-12 18:04:47] frigate.app                    INFO    : Camera processor started for FrontPorch1: 244
[2022-10-12 18:04:47] frigate.app                    INFO    : Camera processor started for FrontPorch2: 247
[2022-10-12 18:04:47] frigate.app                    INFO    : Camera processor started for Backyard: 250
[2022-10-12 18:04:47] frigate.app                    INFO    : Capture process started for Driveway1: 258
[2022-10-12 18:04:47] frigate.mqtt                   INFO    : Turning off recordings for FrontPorch1 via mqtt
[2022-10-12 18:04:47] frigate.mqtt                   INFO    : Turning off snapshots for FrontPorch1 via mqtt
[2022-10-12 18:04:47] frigate.mqtt                   INFO    : Turning off detection for FrontPorch1 via mqtt
[2022-10-12 18:04:47] frigate.app                    INFO    : Capture process started for Driveway2: 262
[2022-10-12 18:04:47] frigate.app                    INFO    : Capture process started for FrontPorch1: 264
[2022-10-12 18:04:47] frigate.app                    INFO    : Capture process started for FrontPorch2: 273
[2022-10-12 18:04:47] frigate.app                    INFO    : Capture process started for Backyard: 278
[2022-10-12 18:04:48] ws4py                          INFO    : Using epoll
[2022-10-12 18:05:55] ws4py                          INFO    : Managing websocket [Local => 127.0.0.1:5002 | Remote => 127.0.0.1:49632]
[2022-10-12 18:06:03] ws4py                          INFO    : Terminating websocket [Local => 127.0.0.1:5002 | Remote => 127.0.0.1:49632]
F driver/usb/usb_driver.cc:406] HandleEvent failed. USB transfer error 1 [LibUsbDataInCallback]
Fatal Python error: Aborted
Thread 0x00007fb0bf976700 (most recent call first):
  File "/usr/lib/python3.9/threading.py", line 312 in wait
  File "/usr/lib/python3.9/multiprocessing/queues.py", line 233 in _feed
  File "/usr/lib/python3.9/threading.py", line 892 in run
  File "/usr/lib/python3.9/threading.py", line 954 in _bootstrap_inner
  File "/usr/lib/python3.9/threading.py", line 912 in _bootstrap
Thread 0x00007fb0fc223740 (most recent call first):
  File "/usr/lib/python3.9/selectors.py", line 416 in select
  File "/usr/lib/python3.9/multiprocessing/connection.py", line 936 in wait
  File "/usr/lib/python3.9/multiprocessing/connection.py", line 429 in _poll
  File "/usr/lib/python3.9/multiprocessing/connection.py", line 262 in poll
  File "/usr/lib/python3.9/multiprocessing/queues.py", line 113 in get
  File "/opt/frigate/frigate/edgetpu.py", line 147 in run_detector
  File "/usr/lib/python3.9/multiprocessing/process.py", line 108 in run
  File "/usr/lib/python3.9/multiprocessing/process.py", line 315 in _bootstrap
  File "/usr/lib/python3.9/multiprocessing/popen_fork.py", line 71 in _launch
  File "/usr/lib/python3.9/multiprocessing/popen_fork.py", line 19 in __init__
  File "/usr/lib/python3.9/multiprocessing/context.py", line 277 in _Popen
  File "/usr/lib/python3.9/multiprocessing/context.py", line 224 in _Popen
  File "/usr/lib/python3.9/multiprocessing/process.py", line 121 in start
  File "/opt/frigate/frigate/edgetpu.py", line 220 in start_or_restart
  File "/opt/frigate/frigate/edgetpu.py", line 189 in __init__
  File "/opt/frigate/frigate/app.py", line 217 in start_detectors
  File "/opt/frigate/frigate/app.py", line 360 in start
  File "/opt/frigate/frigate/__main__.py", line 16 in <module>
  File "/usr/lib/python3.9/runpy.py", line 87 in _run_code
  File "/usr/lib/python3.9/runpy.py", line 197 in _run_module_as_main
[2022-10-12 18:06:08] frigate.watchdog               INFO    : Detection appears to have stopped. Exiting frigate...
[2022-10-12 18:06:08] frigate.app                    INFO    : Stopping...
[2022-10-12 18:06:08] ws4py                          INFO    : Closing all websockets with [1001] 'Server is shutting down'
[2022-10-12 18:06:08] frigate.stats                  INFO    : Exiting watchdog...
[2022-10-12 18:06:08] frigate.events                 INFO    : Exiting event cleanup...
[2022-10-12 18:06:08] frigate.record                 INFO    : Exiting recording cleanup...
[2022-10-12 18:06:08] frigate.watchdog               INFO    : Exiting watchdog...
[2022-10-12 18:06:08] frigate.record                 INFO    : Exiting recording maintenance...
[2022-10-12 18:06:08] frigate.object_processing      INFO    : Exiting object processor...
[2022-10-12 18:06:13] frigate.events                 INFO    : Exiting event processor...
[2022-10-12 18:06:13] peewee.sqliteq                 INFO    : writer received shutdown request, exiting.
[2022-10-12 18:06:13] root                           INFO    : Waiting for detection process to exit gracefully...
/usr/lib/python3.9/multiprocessing/resource_tracker.py:216: UserWarning: resource_tracker: There appear to be 19 leaked shared_memory objects to clean up at shutdown
  warnings.warn('resource_tracker: There appear to be %d '
[cmd] python3 exited 0
[cont-finish.d] executing container finish scripts...
[cont-finish.d] done.
[s6-finish] waiting for services.
[s6-finish] sending all processes the TERM signal.

Operating system

Proxmox

Install method

HassOS Addon

Coral version

USB

Any other information that may be helpful

I have tried moving from a USB 2 and installing a USB 3 PCIE card along with running it with a powered USB 3 hub. This is running HASSIO (Home Assistant 2022.10.3 - Supervisor 2022.10.0 - Operating System 9.2 - Frontend 20221010.0 - latest) under Proxmox 7.2-7 VM and I have passed though the full PCIE slot and verified that it seems to be working. It is running on an older Dell R710 server and virtualization seems to be working fine and being passed through the VM.

NickM-27 commented 2 years ago

Maybe take a look at https://github.com/google-coral/edgetpu/issues/99#issuecomment-719028859

I'd also recommend looking at your BIOS for any USB related options that might be affecting this.

Unfortunately, I don't think there's much we can do as it's something with the host / VM cutting access to the coral.

NickM-27 commented 2 years ago

Also similar to https://github.com/blakeblackshear/frigate/issues/4056

Dutch317 commented 2 years ago

Thanks Nick - I missed that second post. Nothing in the bios as far as USB is concerned or in power settings. I'm thinking you are right, it is something between the host and VM. It seemed to be the same issue by passing the card or just the USB port in Proxmox. For now I'll just switch back to CPU detectors and keep working on the host side to see if there is anything to be done. I have been seeing others running on bare metal with or without Frigate and having the same problems.

tyfoon commented 2 years ago

Have you been able to fix this? I have docker on ubuntu and my USB TPU keeps stopping after a few mins.

Dutch317 commented 2 years ago

No, I really haven't had the time lately to dig into it more. I sure my problem lies between the hardware/Proxmox and passing the usb through correctly. I think the OS is trying to access the USB at times causing the "lock up" and I think it has to do with blacklisting the TPU/USB drivers in the host OS.

renaiku commented 2 years ago

I have the same issue with Home Assistant Yellow

transfer on tag 2 failed. Abort. Deadline exceeded: USB transfer error 2 [LibUsbDataOutCallback]
Fatal Python error: Aborted
Thread 0x0000007f88ad41e0 (most recent call first):
  File "/usr/lib/python3.9/threading.py", line 312 in wait
  File "/usr/lib/python3.9/multiprocessing/queues.py", line 233 in _feed
  File "/usr/lib/python3.9/threading.py", line 892 in run
  File "/usr/lib/python3.9/threading.py", line 954 in _bootstrap_inner
  File "/usr/lib/python3.9/threading.py", line 912 in _bootstrap
Thread 0x0000007f950a0010 (most recent call first):
  File "/usr/lib/python3/dist-packages/tflite_runtime/interpreter.py", line 833 in invoke
  File "/opt/frigate/frigate/edgetpu.py", line 83 in detect_raw
  File "/opt/frigate/frigate/edgetpu.py", line 159 in run_detector
  File "/usr/lib/python3.9/multiprocessing/process.py", line 108 in run
  File "/usr/lib/python3.9/multiprocessing/process.py", line 315 in _bootstrap
  File "/usr/lib/python3.9/multiprocessing/popen_fork.py", line 71 in _launch
  File "/usr/lib/python3.9/multiprocessing/popen_fork.py", line 19 in __init__
  File "/usr/lib/python3.9/multiprocessing/context.py", line 277 in _Popen
  File "/usr/lib/python3.9/multiprocessing/context.py", line 224 in _Popen
  File "/usr/lib/python3.9/multiprocessing/process.py", line 121 in start
  File "/opt/frigate/frigate/edgetpu.py", line 220 in start_or_restart
  File "/opt/frigate/frigate/edgetpu.py", line 189 in __init__
  File "/opt/frigate/frigate/app.py", line 217 in start_detectors
  File "/opt/frigate/frigate/app.py", line 360 in start
  File "/opt/frigate/frigate/__main__.py", line 16 in <module>
  File "/usr/lib/python3.9/runpy.py", line 87 in _run_code
  File "/usr/lib/python3.9/runpy.py", line 197 in _run_module_as_main

With the complete config being:

mqtt:
  host: ****
  port: ****
  topic_prefix: frigate
  user: ****
  password: ****

detectors:
  coral:
    type: edgetpu
    device: usb

cameras:
  tapo_c210:
    ffmpeg:
      # hwaccel_args: -c:v h264_v4l2m2m
      # input_args: -rtsp_transport tcp
      inputs:
        - path: rtsp://user:pwd@ip/stream1
          roles:
            - detect

If I change the detector with

  cpu1:
    type: cpu
    num_threads: 3

everything works fine.

NickM-27 commented 2 years ago

@renaiku are you using a powered usb hub? I doubt the HA yellow can supply enough power for the coral on its own

renaiku commented 2 years ago

@NickM-27 Yes, the one advised by the documentation. Only the coral is plugged on it.

renaiku commented 2 years ago

@NickM-27 Changed official coral usb cable with another one. It works now.

What is weird is that the offical cable works on rpi4, not on my Yellow.

NickM-27 commented 2 years ago

I had a lot of issues with the included cable back when I ran USB coral

alsFC commented 2 years ago

I'm facing the same problem on the Yellow. It was running stable on the Blue with the original USB cable and without extra power supply.

github-actions[bot] commented 1 year ago

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

austwhite commented 1 year ago

@alsFC The Blue can supply more power to USB ports than the HA Yellow can as the HA Yellow is based on the Raspberry Pi CM4 Also, if you have a Yellow using a CM4 with Wifi and Bluetooth capabilities, they seem to interfere with the Coral. The BLue (Odroid) doesn't have WiFi at all. I've seen success by using a long USB cable, connecting to a powered hub significant distance from the Yellow. Someone reported needign 2 metres, but I think that is excessive. I got mine working about 50cm away, which is about 20 inches, give or take. Make sure it connects to a powered USB hub as the Coral needs a bit of juice to work.