search

blakeblackshear / frigate

NVR with realtime local object detection for IP cameras
https://frigate.video
MIT License
18.89k stars 1.72k forks source link

[Detector Support]: going from CPU dectector to Coral is not working #7490

Closed bajwamar closed 1 year ago

bajwamar commented 1 year ago

Describe the problem you are having

I have Frigate working and recording events with CPU dectector (only have 1 Camera and 1 Doorbell Camera) and recently bought the Coral USB and made the change to config file to detectors: coral: type: edgetpu device: usb and now Frigate is not working but if I change the detectors back to CPU it works fine

Version

0.12.1

Frigate config file

mqtt:
  enabled: True
  host: 192.168.1.151
  user: mqtt_user
  password: PASSWORD
  port: 1883

cameras:
  Frigate_Garage_Cam:
    ffmpeg:
      # hwaccel_args: preset-rpi-64-h264
      # input_args: preset-rtsp-generic
      inputs:
        - path: rtsp://USERNAME:PASSWORD@<IP_ADDRESS>/live0
          roles:
            - record

    #      output_args:
    #        record: preset-record-generic-audio-copy

    detect:
      enabled: False
      width: 1920
      height: 1080
      fps: 10
      stationary:
        interval: 0
        threshold: 30

    record:
      enabled: True
      retain:
        days: 7
        mode: motion

    motion:
      mask:
        - 1920,0,1920,242,1920,424,1080,357,0,345,0,0

  Frigate_Doorbell:
    ffmpeg:
      inputs:
        - path: rtsp://<IP_ADDRESS>/MODEL#
          roles:
            - record
    record:
      enabled: True
      retain:
        days: 14
        mode: all

  Frigate_Outdoor_Front1:
    ffmpeg:
      inputs:
        - path: rtsp://USERNAME:PASSWORD@<IP_ADDRESS>/h264Preview_01_sub
          roles:
            - record
    record:
      enabled: True
      retain:
        days: 7
        mode: motion

detectors:
  coral:
    type: edgetpu
    device: usb

logger:
  default: critical

docker-compose file or Docker CLI command

n/a

Relevant log output

s6-rc: info: service legacy-cont-init: starting
s6-rc: info: service legacy-cont-init successfully started
s6-rc: info: service log-prepare: starting
s6-rc: info: service log-prepare successfully started
s6-rc: info: service nginx-log: starting
s6-rc: info: service go2rtc-log: starting
s6-rc: info: service frigate-log: starting
s6-rc: info: service go2rtc-log successfully started
s6-rc: info: service go2rtc: starting
s6-rc: info: service nginx-log successfully started
s6-rc: info: service frigate-log successfully started
s6-rc: info: service go2rtc successfully started
s6-rc: info: service go2rtc-healthcheck: starting
s6-rc: info: service frigate: starting
s6-rc: info: service go2rtc-healthcheck successfully started
s6-rc: info: service frigate successfully started
s6-rc: info: service nginx: starting
2023-08-16 15:33:30.092080165  [INFO] Preparing go2rtc config...
2023-08-16 15:33:30.100197060  [INFO] Starting Frigate...
s6-rc: info: service nginx successfully started
s6-rc: info: service legacy-services: starting
2023-08-16 15:33:30.128248529  [INFO] Starting NGINX...
s6-rc: info: service legacy-services successfully started
2023-08-16 15:33:30.290024451  [INFO] Got IP address from supervisor: 192.168.1.151
2023-08-16 15:33:30.415422752  2023/08/16 09:33:30 [error] 136#136: *1 connect() failed (111: Connection refused) while connecting to upstream, client: 172.30.32.1, server: , request: "GET /api/stats HTTP/1.1", upstream: "http://127.0.0.1:5001/stats", host: "ccab4aaf-frigate-fa:5000"
2023-08-16 15:33:30.416814623  172.30.32.1 - - [16/Aug/2023:09:33:30 -0600] "GET /api/stats HTTP/1.1" 502 157 "-" "HomeAssistant/2023.8.2 aiohttp/3.8.5 Python/3.11" "-"
2023-08-16 15:33:30.456975639  [INFO] Got WebRTC port from supervisor: 8555
2023-08-16 15:33:31.324567621  [INFO] Starting go2rtc...
2023-08-16 15:33:31.550056608  09:33:31.549 INF go2rtc version 1.2.0 linux/arm64
2023-08-16 15:33:31.551632699  09:33:31.551 INF [api] listen addr=:1984
2023-08-16 15:33:31.552753833  09:33:31.552 INF [rtsp] listen addr=:8554
2023-08-16 15:33:31.554307331  09:33:31.554 INF [srtp] listen addr=:8443
2023-08-16 15:33:31.555545130  09:33:31.555 INF [webrtc] listen addr=:8555
2023-08-16 15:33:34.635772293  [2023-08-16 09:33:34] frigate.app                    INFO    : Starting Frigate (0.12.1-367d724)
2023-08-16 15:33:34.740395554  [2023-08-16 09:33:34] frigate.app                    INFO    : Creating directory: /tmp/cache
2023-08-16 15:33:34.987199608  Process detector:coral:
2023-08-16 15:33:34.993143217  Traceback (most recent call last):
2023-08-16 15:33:34.993337788    File "/usr/lib/python3/dist-packages/tflite_runtime/interpreter.py", line 160, in load_delegate
2023-08-16 15:33:34.993340640      delegate = Delegate(library, options)
2023-08-16 15:33:34.993381047    File "/usr/lib/python3/dist-packages/tflite_runtime/interpreter.py", line 119, in __init__
2023-08-16 15:33:34.993384658      raise ValueError(capture.message)
2023-08-16 15:33:34.995178987  ValueError
2023-08-16 15:33:34.995191524  
2023-08-16 15:33:34.995194357  During handling of the above exception, another exception occurred:
2023-08-16 15:33:34.995195487  
2023-08-16 15:33:34.995197079  Traceback (most recent call last):
2023-08-16 15:33:34.995199283    File "/usr/lib/python3.9/multiprocessing/process.py", line 315, in _bootstrap
2023-08-16 15:33:34.995200561      self.run()
2023-08-16 15:33:34.995202616    File "/usr/lib/python3.9/multiprocessing/process.py", line 108, in run
2023-08-16 15:33:34.995305559      self._target(*self._args, **self._kwargs)
2023-08-16 15:33:34.995310300    File "/opt/frigate/frigate/object_detection.py", line 98, in run_detector
2023-08-16 15:33:34.995312707      object_detector = LocalObjectDetector(detector_config=detector_config)
2023-08-16 15:33:34.995314892    File "/opt/frigate/frigate/object_detection.py", line 52, in __init__
2023-08-16 15:33:34.995316763      self.detect_api = create_detector(detector_config)
2023-08-16 15:33:34.995413132    File "/opt/frigate/frigate/detectors/__init__.py", line 24, in create_detector
2023-08-16 15:33:34.995416169      return api(detector_config)
2023-08-16 15:33:34.995468483    File "/opt/frigate/frigate/detectors/plugins/edgetpu_tfl.py", line 37, in __init__
2023-08-16 15:33:34.995471576      edge_tpu_delegate = load_delegate("libedgetpu.so.1.0", device_config)
2023-08-16 15:33:34.995474261    File "/usr/lib/python3/dist-packages/tflite_runtime/interpreter.py", line 162, in load_delegate
2023-08-16 15:33:34.995476279      raise ValueError('Failed to load delegate from {}\n{}'.format(
2023-08-16 15:33:34.995478224  ValueError: Failed to load delegate from libedgetpu.so.1.0
2023-08-16 15:33:34.995479279  
2023-08-16 15:33:35.412135647  2023/08/16 09:33:35 [error] 136#136: *1 connect() failed (111: Connection refused) while connecting to upstream, client: 172.30.32.1, server: , request: "GET /api/stats HTTP/1.1", upstream: "http://127.0.0.1:5001/stats", host: "ccab4aaf-frigate-fa:5000"
2023-08-16 15:33:35.416446665  172.30.32.1 - - [16/Aug/2023:09:33:35 -0600] "GET /api/stats HTTP/1.1" 502 157 "-" "HomeAssistant/2023.8.2 aiohttp/3.8.5 Python/3.11" "-"
2023-08-16 15:33:40.090403544  [INFO] Starting go2rtc healthcheck service...
2023-08-16 15:33:43.752896818  172.30.32.1 - - [16/Aug/2023:09:33:43 -0600] "GET /api/stats HTTP/1.1" 200 2592 "-" "HomeAssistant/2023.8.2 aiohttp/3.8.5 Python/3.11" "-"
2023-08-16 15:33:51.722683012  172.30.32.1 - - [16/Aug/2023:09:33:51 -0600] "GET /api/stats HTTP/1.1" 200 2561 "-" "HomeAssistant/2023.8.2 aiohttp/3.8.5 Python/3.11" "-"
s6-rc: info: service legacy-services: stopping
s6-rc: info: service legacy-services successfully stopped
s6-rc: info: service nginx: stopping
s6-rc: info: service go2rtc-healthcheck: stopping
2023-08-16 15:33:55.926145876  [INFO] The go2rtc-healthcheck service exited with code 256 (by signal 15)
s6-rc: info: service go2rtc-healthcheck successfully stopped
2023-08-16 15:33:56.004253518  [INFO] Service NGINX exited with code 0 (by signal 0)
s6-rc: info: service nginx successfully stopped
s6-rc: info: service nginx-log: stopping
s6-rc: info: service frigate: stopping
s6-rc: info: service nginx-log successfully stopped
2023-08-16 15:33:58.404102643  /usr/lib/python3.9/multiprocessing/resource_tracker.py:216: UserWarning: resource_tracker: There appear to be 35 leaked shared_memory objects to clean up at shutdown
2023-08-16 15:33:58.404114847    warnings.warn('resource_tracker: There appear to be %d '
2023-08-16 15:33:58.420215940  [INFO] Service Frigate exited with code 0 (by signal 0)
s6-rc: info: service frigate successfully stopped
s6-rc: info: service go2rtc: stopping
s6-rc: info: service frigate-log: stopping
2023-08-16 15:33:58.437950957  exit OK
s6-rc: info: service frigate-log successfully stopped
2023-08-16 15:33:58.461111610  [INFO] The go2rtc service exited with code 0 (by signal 0)
s6-rc: info: service go2rtc successfully stopped
s6-rc: info: service go2rtc-log: stopping
s6-rc: info: service go2rtc-log successfully stopped
s6-rc: info: service log-prepare: stopping
s6-rc: info: service s6rc-fdholder: stopping
s6-rc: info: service log-prepare successfully stopped
s6-rc: info: service legacy-cont-init: stopping
s6-rc: info: service s6rc-fdholder successfully stopped
s6-rc: info: service legacy-cont-init successfully stopped
s6-rc: info: service fix-attrs: stopping
s6-rc: info: service fix-attrs successfully stopped
s6-rc: info: service s6rc-oneshot-runner: stopping
s6-rc: info: service s6rc-oneshot-runner successfully stopped

Operating system

HassOS

Install method

HassOS Addon

Coral version

USB

Any other information that may be helpful

running lsusb shows the following info Bus 001 Device 001: ID 1d6b:0002 Bus 001 Device 002: ID 2109:3431 Bus 002 Device 002: ID 174c:55aa Bus 002 Device 001: ID 1d6b:0003 Bus 001 Device 003: ID 10c4:3a60

I have an SSD connected to USB 3.0 I have SkyConnect connected to USB 2.0 (I think Bus 001 Device 003: ID 10c4:3a60 is the skyconnect)

NickM-27 commented 1 year ago

This is likely a power issue where the USB ports are not able to supply enough power for all devices, a powered USB hub is generally recommended especially for devices like RPi

bajwamar commented 1 year ago

do you know a good powered USB hub that would work with RPi 4 and Coral?

NickM-27 commented 1 year ago

There is one linked in the docs https://docs.frigate.video/frigate/installation/#raspberry-pi-34

hackneyb commented 1 year ago

There is one linked in the docs https://docs.frigate.video/frigate/installation/#raspberry-pi-34

I happened to buy this one by chance and can attest it works. I'm sure it ended up in your documentation for a good reason.

NickM-27 commented 1 year ago

There is one linked in the docs https://docs.frigate.video/frigate/installation/#raspberry-pi-34

I happened to buy this one by chance and can attest it works. I'm sure it ended up in your documentation for a good reason.

Just to clarify, they are not my docs 👍

bajwamar commented 1 year ago

so I got the USB hub but still frigate is not running (below is the log and it looks identical to before)

s6-rc: info: service legacy-cont-init: starting
s6-rc: info: service legacy-cont-init successfully started
s6-rc: info: service log-prepare: starting
s6-rc: info: service log-prepare successfully started
s6-rc: info: service nginx-log: starting
s6-rc: info: service go2rtc-log: starting
s6-rc: info: service frigate-log: starting
s6-rc: info: service go2rtc-log successfully started
s6-rc: info: service go2rtc: starting
s6-rc: info: service frigate-log successfully started
s6-rc: info: service nginx-log successfully started
s6-rc: info: service go2rtc successfully started
s6-rc: info: service go2rtc-healthcheck: starting
s6-rc: info: service frigate: starting
s6-rc: info: service go2rtc-healthcheck successfully started
s6-rc: info: service frigate successfully started
s6-rc: info: service nginx: starting
2023-08-17 23:52:17.059456078  [INFO] Preparing go2rtc config...
2023-08-17 23:52:17.069806847  [INFO] Starting Frigate...
s6-rc: info: service nginx successfully started
s6-rc: info: service legacy-services: starting
2023-08-17 23:52:17.097714282  [INFO] Starting NGINX...
s6-rc: info: service legacy-services successfully started
2023-08-17 23:52:17.258330953  [INFO] Got IP address from supervisor: 192.168.1.151
2023-08-17 23:52:17.405311523  [INFO] Got WebRTC port from supervisor: 8555
2023-08-17 23:52:18.476154432  [INFO] Starting go2rtc...
2023-08-17 23:52:18.710212064  17:52:18.709 INF go2rtc version 1.2.0 linux/arm64
2023-08-17 23:52:18.711326385  17:52:18.711 INF [api] listen addr=:1984
2023-08-17 23:52:18.712458612  17:52:18.712 INF [rtsp] listen addr=:8554
2023-08-17 23:52:18.713519304  17:52:18.713 INF [srtp] listen addr=:8443
2023-08-17 23:52:18.714401534  17:52:18.714 INF [webrtc] listen addr=:8555
2023-08-17 23:52:22.453381981  2023/08/17 17:52:22 [error] 137#137: *1 connect() failed (111: Connection refused) while connecting to upstream, client: 172.30.32.1, server: , request: "GET /api/stats HTTP/1.1", upstream: "http://127.0.0.1:5001/stats", host: "ccab4aaf-frigate-fa:5000"
2023-08-17 23:52:22.453737903  172.30.32.1 - - [17/Aug/2023:17:52:22 -0600] "GET /api/stats HTTP/1.1" 502 157 "-" "HomeAssistant/2023.8.2 aiohttp/3.8.5 Python/3.11" "-"
2023-08-17 23:52:23.952213512  [2023-08-17 17:52:23] frigate.app                    INFO    : Starting Frigate (0.12.1-367d724)
2023-08-17 23:52:24.077938607  [2023-08-17 17:52:24] frigate.app                    INFO    : Creating directory: /tmp/cache
2023-08-17 23:52:27.074386574  [INFO] Starting go2rtc healthcheck service...
2023-08-17 23:52:30.826772911  172.30.32.1 - - [17/Aug/2023:17:52:30 -0600] "GET /api/stats HTTP/1.1" 200 2653 "-" "HomeAssistant/2023.8.2 aiohttp/3.8.5 Python/3.11" "-"
2023-08-17 23:52:38.946056942  172.30.32.1 - - [17/Aug/2023:17:52:38 -0600] "GET /api/stats HTTP/1.1" 200 2590 "-" "HomeAssistant/2023.8.2 aiohttp/3.8.5 Python/3.11" "-"
2023-08-17 23:52:46.737062914  172.30.32.1 - - [17/Aug/2023:17:52:46 -0600] "GET /api/stats HTTP/1.1" 200 2658 "-" "HomeAssistant/2023.8.2 aiohttp/3.8.5 Python/3.11" "-"
2023-08-17 23:52:50.231178085  Process detector:coral:
2023-08-17 23:52:50.246557536  Traceback (most recent call last):
2023-08-17 23:52:50.246574795    File "/usr/lib/python3/dist-packages/tflite_runtime/interpreter.py", line 160, in load_delegate
2023-08-17 23:52:50.246623776      delegate = Delegate(library, options)
2023-08-17 23:52:50.246663442    File "/usr/lib/python3/dist-packages/tflite_runtime/interpreter.py", line 119, in __init__
2023-08-17 23:52:50.246668220      raise ValueError(capture.message)
2023-08-17 23:52:50.247130122  ValueError
2023-08-17 23:52:50.247173066  
2023-08-17 23:52:50.247207010  During handling of the above exception, another exception occurred:
2023-08-17 23:52:50.247213713  
2023-08-17 23:52:50.247215343  Traceback (most recent call last):
2023-08-17 23:52:50.247219436    File "/usr/lib/python3.9/multiprocessing/process.py", line 315, in _bootstrap
2023-08-17 23:52:50.247220787      self.run()
2023-08-17 23:52:50.247222917    File "/usr/lib/python3.9/multiprocessing/process.py", line 108, in run
2023-08-17 23:52:50.247732244      self._target(*self._args, **self._kwargs)
2023-08-17 23:52:50.247862965    File "/opt/frigate/frigate/object_detection.py", line 98, in run_detector
2023-08-17 23:52:50.247878743      object_detector = LocalObjectDetector(detector_config=detector_config)
2023-08-17 23:52:50.247880946    File "/opt/frigate/frigate/object_detection.py", line 52, in __init__
2023-08-17 23:52:50.248201739      self.detect_api = create_detector(detector_config)
2023-08-17 23:52:50.248214313    File "/opt/frigate/frigate/detectors/__init__.py", line 24, in create_detector
2023-08-17 23:52:50.248217554      return api(detector_config)
2023-08-17 23:52:50.248219868    File "/opt/frigate/frigate/detectors/plugins/edgetpu_tfl.py", line 37, in __init__
2023-08-17 23:52:50.248257998      edge_tpu_delegate = load_delegate("libedgetpu.so.1.0", device_config)
2023-08-17 23:52:50.249042285    File "/usr/lib/python3/dist-packages/tflite_runtime/interpreter.py", line 162, in load_delegate
2023-08-17 23:52:50.249055692      raise ValueError('Failed to load delegate from {}\n{}'.format(
2023-08-17 23:52:50.249057766  ValueError: Failed to load delegate from libedgetpu.so.1.0
2023-08-17 23:52:50.249062988  
2023-08-17 23:52:54.959127490  172.30.32.1 - - [17/Aug/2023:17:52:54 -0600] "GET /api/stats HTTP/1.1" 200 2592 "-" "HomeAssistant/2023.8.2 aiohttp/3.8.5 Python/3.11" "-"
s6-rc: info: service legacy-services: stopping
s6-rc: info: service legacy-services successfully stopped
s6-rc: info: service nginx: stopping
s6-rc: info: service go2rtc-healthcheck: stopping
2023-08-17 23:52:56.216984962  [INFO] The go2rtc-healthcheck service exited with code 256 (by signal 15)
s6-rc: info: service go2rtc-healthcheck successfully stopped
2023-08-17 23:52:56.257726415  [INFO] Service NGINX exited with code 0 (by signal 0)
s6-rc: info: service nginx successfully stopped
s6-rc: info: service nginx-log: stopping
s6-rc: info: service frigate: stopping
s6-rc: info: service nginx-log successfully stopped
2023-08-17 23:52:58.534167896  /usr/lib/python3.9/multiprocessing/resource_tracker.py:216: UserWarning: resource_tracker: There appear to be 28 leaked shared_memory objects to clean up at shutdown
2023-08-17 23:52:58.534188748    warnings.warn('resource_tracker: There appear to be %d '
2023-08-17 23:52:58.552267186  [INFO] Service Frigate exited with code 0 (by signal 0)
s6-rc: info: service frigate successfully stopped
s6-rc: info: service go2rtc: stopping
s6-rc: info: service frigate-log: stopping
s6-rc: info: service frigate-log successfully stopped
2023-08-17 23:52:58.580175770  exit OK
2023-08-17 23:52:58.609044805  [INFO] The go2rtc service exited with code 0 (by signal 0)
s6-rc: info: service go2rtc successfully stopped
s6-rc: info: service go2rtc-log: stopping
s6-rc: info: service go2rtc-log successfully stopped
s6-rc: info: service log-prepare: stopping
s6-rc: info: service s6rc-fdholder: stopping
s6-rc: info: service log-prepare successfully stopped
s6-rc: info: service legacy-cont-init: stopping
s6-rc: info: service s6rc-fdholder successfully stopped
s6-rc: info: service legacy-cont-init successfully stopped
s6-rc: info: service fix-attrs: stopping
s6-rc: info: service fix-attrs successfully stopped
s6-rc: info: service s6rc-oneshot-runner: stopping
s6-rc: info: service s6rc-oneshot-runner successfully stopped
NickM-27 commented 1 year ago

How are the usb devices setup with the hub?

What does running lsusb inside the container show?

bajwamar commented 1 year ago

running lsusb shows the following info Bus 002 Device 004: ID 1a6e:089a Bus 002 Device 002: ID 05e3:0626 Bus 001 Device 001: ID 1d6b:0002 Bus 001 Device 003: ID 05e3:0610 Bus 001 Device 002: ID 2109:3431 Bus 002 Device 002: ID 174c:55aa Bus 002 Device 001: ID 1d6b:0003 Bus 001 Device 003: ID 10c4:3a60

NickM-27 commented 1 year ago

Is that inside the container (addon)?

bajwamar commented 1 year ago

I have HA running on RPi 4 and frigate addon

NickM-27 commented 1 year ago

You can run docker exec -it addon_ccab4aaf_frigate lsusb to se what it shows in the container.

bajwamar commented 1 year ago

had to change the code to docker exec -it addon_ccab4aaf_frigate-**fa** lsusb but for Error OCI runtime exec failed: exec failed: unable to start container process: exec: "lsusb": executable file not found in $PATH: unknown

NickM-27 commented 1 year ago

Oh, right, so it'll need to be

docker exec -it addon_ccab4aaf_frigate-fa apt-get update && apt-get install lsusb -y && lsusb

bajwamar commented 1 year ago

now it says command not found: apt-get

NickM-27 commented 1 year ago

docker exec -it addon_ccab4aaf_frigate-fa apt update && apt install lsusb -y && lsusb

bajwamar commented 1 year ago

was able to get the addon up and running with Coral had to disable the protection mode within the addon but I still have high CPU usable

NickM-27 commented 1 year ago

Show a screenshot of the system page.

bajwamar commented 1 year ago

SmartSelect_20230817_193743_Home Assistant SmartSelect_20230817_193706_Home Assistant

NickM-27 commented 1 year ago

Sorry, I meant the system tab in the frigate webUI.

bajwamar commented 1 year ago

SmartSelect_20230817_195928_Home Assistant SmartSelect_20230817_195916_Home Assistant

NickM-27 commented 1 year ago

I'd definitely suggest leaving detect fps at 5, you're also going to want to enable hardware acceleration

bajwamar commented 1 year ago

What kind of hardware acceleration should I enable for rpi4 with coral

NickM-27 commented 1 year ago

preset-rpi-64-h264

bajwamar commented 1 year ago

so I have setup the frigate.yaml file accordingly and I was reading the documentation and it mentioned that I should also allocate more mem for GPU but these commond mentioned in https://docs.frigate.video/configuration/hardware_acceleration#raspberry-pi-34 don't work I have disabled the protection mode on the Advance SSH & Web Terminal it says command not found: raspi-config

NickM-27 commented 1 year ago

In this case for ha os you need to edit /mnt/boot/config.txt

https://gist.github.com/enegaard/a57af286205914bd912270c89650fb1b

bajwamar commented 1 year ago

I was able to get into /mnt directory but there is no /boot directory image

NickM-27 commented 1 year ago

Odd, I'm not sure unfortunately they don't make it easy in ha os. Is hwaccel working as is?

bajwamar commented 1 year ago

image

NickM-27 commented 1 year ago

That's expected for RPi, like it says that does not indicate any problems

bajwamar commented 1 year ago

still have high usage on CPU for ffmpeg image

NickM-27 commented 1 year ago

Yeah, unfortunately that's not unexpected for a Pi running detect on a 1920x1080 stream.

NickM-27 commented 1 year ago

Could try -c:v h264_v4l2m2m as hwaccel args and see if it makes a difference.

github-actions[bot] commented 1 year ago

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.