Closed vchrizz closed 11 months ago
I tried to add the pycoral example repository as volume, but can not run the test example within the docker container, like I am able on the host:
# docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
32b6463de52d roflcoopter/viseron:latest "/init" 59 seconds ago Up 57 seconds 0.0.0.0:8888->8888/tcp, :::8888->8888/tcp viseron
# docker exec -it 32 python3 /root/pycoral/examples/classify_image.py --model /root/pycoral/test_data/mobilenet_v2_1.0_224_inat_bird_quant_edgetpu.tflite --labels /root/pycoral/test_data/inat_bird_labels.txt --input /root/pycoral/test_data/parrot.jpg
Traceback (most recent call last):
File "/root/pycoral/examples/classify_image.py", line 121, in <module>
main()
File "/root/pycoral/examples/classify_image.py", line 79, in main
image = Image.open(args.input).convert('RGB').resize(size, Image.ANTIALIAS)
AttributeError: module 'PIL.Image' has no attribute 'ANTIALIAS'
seems like the container is missing something..
reading How To Fix AttributeError: module ‘PIL.Image’ has no attribute ‘ANTIALIAS’ it seems to be the Pillow
package, installed via pip..
# docker exec -it 32b6463de52d pip list | grep Pillow
Pillow 10.0.1
# docker exec -it 32b6463de52d pip install Pillow==9.5.0
Collecting Pillow==9.5.0
Downloading Pillow-9.5.0-cp38-cp38-manylinux_2_28_x86_64.whl (3.4 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 3.4/3.4 MB 11.0 MB/s eta 0:00:00
Installing collected packages: Pillow
Attempting uninstall: Pillow
Found existing installation: Pillow 10.0.1
Uninstalling Pillow-10.0.1:
Successfully uninstalled Pillow-10.0.1
Successfully installed Pillow-9.5.0
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv
[notice] A new release of pip is available: 23.2.1 -> 23.3.1
[notice] To update, run: python3 -m pip install --upgrade pip
now the example works (but even after restart of viseron it still does not find the edgetpu):
# docker exec -it 32b6463de52d python3 /root/pycoral/examples/classify_image.py --model /root/pycoral/test_data/mobilenet_v2_1.0_224_inat_bird_quant_edgetpu.tflite --labels /root/pycoral/test_data/inat_bird_labels.txt --input /root/pycoral/test_data/parrot.jpg
/root/pycoral/examples/classify_image.py:79: DeprecationWarning: ANTIALIAS is deprecated and will be removed in Pillow 10 (2023-07-01). Use LANCZOS or Resampling.LANCZOS instead.
image = Image.open(args.input).convert('RGB').resize(size, Image.ANTIALIAS)
----INFERENCE TIME----
Note: The first inference on Edge TPU is slow because it includes loading the model into Edge TPU memory.
109.4ms
13.0ms
12.9ms
13.3ms
12.9ms
-------RESULTS--------
Ara macao (Scarlet Macaw): 0.75781
https://www.jeffgeerling.com/blog/2023/testing-coral-tpu-accelerator-m2-or-pcie-docker
Thanks for your report!
Viseron does not use Pillow so thats not the issue here, but it is interesting that the example works and Viserons implementation does not. Can you restart Viseron and provide the startup logs?
complete log after restart:
[s6-init] making user provided files available at /var/run/s6/etc...exited 0.
[s6-init] ensuring user provided files have correct perms...exited 0.
[fix-attrs.d] applying ownership & permissions fixes...
[fix-attrs.d] done.
[cont-init.d] executing container initialization scripts...
[cont-init.d] 10-adduser: executing...
************************ UID/GID *************************
User uid: 1000
User gid: 1000
************************** Done **************************
[cont-init.d] 10-adduser: exited 0.
[cont-init.d] 20-gid-video-device: executing...
[cont-init.d] 20-gid-video-device: exited 0.
[cont-init.d] 30-edgetpu-permission: executing...
************** Setting EdgeTPU permissions ***************
Coral Vendor IDs:
"1a6e"
"18d1"
Matching USB devices:
Bus 002 Device 002: ID 18d1:9302
Adding user to group plugdev
No EdgeTPU PCI device was found
************************** Done **************************
[cont-init.d] 30-edgetpu-permission: exited 0.
[cont-init.d] 40-set-env-vars: executing...
****** Checking for hardware acceleration platforms ******
OpenCL cannot be used
VA-API cannot be used
CUDA cannot be used
*********************** Done *****************************
[cont-init.d] 40-set-env-vars: exited 0.
[cont-init.d] 50-check-if-rpi: executing...
********** Checking if we are running on an RPi **********
Not running on any supported RPi
*********************** Done *****************************
[cont-init.d] 50-check-if-rpi: exited 0.
[cont-init.d] 55-check-if-jetson: executing...
****** Checking if we are running on a Jetson Board ******
Not running on any supported Jetson board
*********************** Done *****************************
[cont-init.d] 55-check-if-jetson: exited 0.
[cont-init.d] 60-ffmpeg-path: executing...
****************** Getting FFmpeg path *******************
FFmpeg path: /home/abc/bin/ffmpeg
*********************** Done *****************************
[cont-init.d] 60-ffmpeg-path: exited 0.
[cont-init.d] 70-gstreamer-path: executing...
***************** Getting GStreamer path *****************
GStreamer path: /usr/bin/gst-launch-1.0
*********************** Done *****************************
[cont-init.d] 70-gstreamer-path: exited 0.
[cont-init.d] done.
[services.d] starting services
[services.d] done.
[2023-10-23 14:29:51] [INFO ] [viseron.core] - -------------------------------------------
[2023-10-23 14:29:51] [INFO ] [viseron.core] - Initializing Viseron
[2023-10-23 14:29:51] [INFO ] [viseron.components] - Setting up component logger
[2023-10-23 14:29:51] [INFO ] [viseron.components] - Setup of component logger took 0.0 seconds
[2023-10-23 14:29:51] [INFO ] [viseron.components] - Setting up component data_stream
[2023-10-23 14:29:51] [INFO ] [viseron.components] - Setup of component data_stream took 0.0 seconds
[2023-10-23 14:29:51] [INFO ] [viseron.components] - Setting up component webserver
[2023-10-23 14:29:51] [INFO ] [viseron.components] - Setup of component webserver took 0.0 seconds
[2023-10-23 14:29:51] [INFO ] [viseron.components] - Setting up component edgetpu
[2023-10-23 14:29:51] [INFO ] [viseron.components] - Setting up component mog2
[2023-10-23 14:29:51] [INFO ] [viseron.components] - Setting up component ffmpeg
[2023-10-23 14:29:51] [INFO ] [viseron.components] - Setting up component darknet
[2023-10-23 14:29:51] [INFO ] [viseron.components] - Setting up component nvr
[2023-10-23 14:29:51] [INFO ] [viseron.components] - Setup of component nvr took 0.0 seconds
[2023-10-23 14:29:51] [INFO ] [viseron.components] - Setup of component mog2 took 0.0 seconds
[2023-10-23 14:29:51] [INFO ] [viseron.components] - Setup of component ffmpeg took 0.0 seconds
[2023-10-23 14:29:51] [DEBUG ] [viseron.components.darknet] - Using OpenCV DNN Darknet
[2023-10-23 14:29:51] [DEBUG ] [viseron.components.darknet] - Using weights /detectors/models/darknet/default.weights and config /detectors/models/darknet/default.cfg, 640x640
[2023-10-23 14:29:51] [DEBUG ] [viseron.components.darknet] - DNN backend: 0
[2023-10-23 14:29:51] [DEBUG ] [viseron.components.darknet] - DNN target: 0
[2023-10-23 14:29:51] [DEBUG ] [viseron.components.edgetpu] - Available devices: [{'type': 'usb', 'path': '/sys/bus/usb/devices/2-1'}]
[2023-10-23 14:29:51] [INFO ] [viseron.components] - Setup of component edgetpu took 0.0 seconds
[2023-10-23 14:29:52] [WARNING ] [viseron.components] - Domain object_detector with identifier camera_1 already in setup queue. Skipping setup of domain object_detector with identifier camera_1 for component darknet
[2023-10-23 14:29:52] [INFO ] [viseron.components] - Setup of component darknet took 0.3 seconds
[2023-10-23 14:29:52] [INFO ] [viseron.components] - Setting up domain camera for component ffmpeg with identifier camera_1
[2023-10-23 14:29:52] [INFO ] [viseron.components] - Setting up domain object_detector for component edgetpu with identifier camera_1
[2023-10-23 14:29:52] [INFO ] [viseron.components] - Setting up domain motion_detector for component mog2 with identifier camera_1
[2023-10-23 14:29:52] [INFO ] [viseron.components] - Setting up domain nvr for component nvr with identifier camera_1
[2023-10-23 14:29:52] [INFO ] [viseron.components] - Setting up domain image_classification for component edgetpu with identifier camera_1
[2023-10-23 14:29:52] [DEBUG ] [viseron.components.ffmpeg.stream.camera_1] - Stream information from FFprobe: Width: 1920 Height: 1080 FPS: 25 Video Codec: h264 Audio Codec: aac
[2023-10-23 14:29:52] [DEBUG ] [viseron.components.ffmpeg.recorder.camera_1] - Running cleanup
[2023-10-23 14:29:52] [DEBUG ] [viseron.components.ffmpeg.recorder.camera_1] - Items in /recordings/camera_1/2023-10-17: 24
[2023-10-23 14:29:52] [DEBUG ] [viseron.components.ffmpeg.recorder.camera_1] - Items in /recordings/camera_1/2023-10-18: 48
[2023-10-23 14:29:52] [DEBUG ] [viseron.components.ffmpeg.recorder.camera_1] - Initializing recorder
[2023-10-23 14:29:52] [DEBUG ] [viseron.components.ffmpeg.camera.camera_1] - Initializing camera MyCam1
[2023-10-23 14:29:52] [DEBUG ] [viseron.components.ffmpeg.camera.camera_1] - Resolution: 1920x1080 @ 25 FPS
[2023-10-23 14:29:52] [DEBUG ] [viseron.components.ffmpeg.camera.camera_1] - Camera MyCam1 initialized
[2023-10-23 14:29:52] [INFO ] [viseron.components] - Setup of domain camera for component ffmpeg with identifier camera_1 took 0.0 seconds
[2023-10-23 14:29:52] [DEBUG ] [viseron.components.edgetpu] - Loading interpreter with device :0, model /detectors/models/edgetpu/mobiledet_model.tflite
[2023-10-23 14:29:52] [INFO ] [viseron.components] - Setup of domain motion_detector for component mog2 with identifier camera_1 took 0.0 seconds
[2023-10-23 14:29:52] [DEBUG ] [viseron.components.edgetpu] - Using labels from /detectors/models/edgetpu/labels.txt
[2023-10-23 14:29:56] [DEBUG ] [viseron.components.edgetpu] - Loading interpreter with device usb, model /classifiers/models/edgetpu/tf2_mobilenet_v3_edgetpu_1.0_224_ptq_edgetpu.tflite
[2023-10-23 14:29:56] [DEBUG ] [viseron.components.edgetpu] - Using labels from /classifiers/models/edgetpu/labels.txt
[2023-10-23 14:30:04] [DEBUG ] [viseron.components.edgetpu.image_classification.camera_1] - Post processor will run for labels: ['person']
[2023-10-23 14:30:04] [INFO ] [viseron.components] - Setup of domain image_classification for component edgetpu with identifier camera_1 took 11.4 seconds
[2023-10-23 14:30:04] [ERROR ] [viseron.components.edgetpu] - Error when trying to load EdgeTPU: Failed to load delegate from libedgetpu.so.1
Process child_process.edgetpu.object_detector.process:
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/tflite_runtime/interpreter.py", line 160, in load_delegate
delegate = Delegate(library, options)
File "/usr/lib/python3/dist-packages/tflite_runtime/interpreter.py", line 119, in __init__
raise ValueError(capture.message)
ValueError
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/src/viseron/components/edgetpu/__init__.py", line 214, in make_interpreter
interpreter = make_interpreter(
File "/usr/lib/python3/dist-packages/pycoral/utils/edgetpu.py", line 87, in make_interpreter
delegates = [load_edgetpu_delegate({'device': device} if device else {})]
File "/usr/lib/python3/dist-packages/pycoral/utils/edgetpu.py", line 52, in load_edgetpu_delegate
return tflite.load_delegate(_EDGETPU_SHARED_LIB, options or {})
File "/usr/lib/python3/dist-packages/tflite_runtime/interpreter.py", line 162, in load_delegate
raise ValueError('Failed to load delegate from {}\n{}'.format(
ValueError: Failed to load delegate from libedgetpu.so.1
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/usr/lib/python3.8/multiprocessing/process.py", line 315, in _bootstrap
self.run()
File "/usr/lib/python3.8/multiprocessing/process.py", line 108, in run
self._target(*self._args, **self._kwargs)
File "/src/viseron/helpers/child_process_worker.py", line 93, in _process_frames
self.process_initialization()
File "/src/viseron/components/edgetpu/__init__.py", line 226, in process_initialization
self.interpreter = self.make_interpreter(self._device, self._model)
File "/src/viseron/components/edgetpu/__init__.py", line 220, in make_interpreter
raise MakeInterpreterError from error
viseron.components.edgetpu.MakeInterpreterError
[2023-10-23 14:30:12] [WARNING ] [viseron.components] - Setup of domain object_detector for component edgetpu with identifier camera_1 is taking longer than 20 seconds
F driver/usb/usb_driver.cc:1148] HandleQueuedBulkIn transfer in failed. Not found: USB transfer error 5 [LibUsbDataInCallback]
[2023-10-23 14:30:34] [ERROR ] [viseron.components.edgetpu] - Failed to load EdgeTPU in child process
NoneType: None
[2023-10-23 14:30:34] [ERROR ] [viseron.components] - Domain object_detector for component edgetpu is not ready. Retrying in 10 seconds. Error:
[2023-10-23 14:30:44] [INFO ] [viseron.components] - Setting up domain object_detector for component edgetpu with identifier camera_1, attempt 2
[2023-10-23 14:30:44] [DEBUG ] [viseron.components.edgetpu] - Loading interpreter with device :0, model /detectors/models/edgetpu/mobiledet_model.tflite
[2023-10-23 14:30:44] [DEBUG ] [viseron.components.edgetpu] - Using labels from /detectors/models/edgetpu/labels.txt
[2023-10-23 14:30:51] [INFO ] [viseron.components] - Setup of domain object_detector for component edgetpu with identifier camera_1 took 6.9 seconds
[2023-10-23 14:30:51] [DEBUG ] [viseron.components.nvr.nvr.camera_1] - Initializing NVR for camera MyCam1
[2023-10-23 14:30:51] [DEBUG ] [viseron.components.nvr.nvr.camera_1] - Waiting for first frame
[2023-10-23 14:30:51] [DEBUG ] [viseron.components.ffmpeg.camera.camera_1] - Starting capture thread
[2023-10-23 14:30:51] [DEBUG ] [viseron.components.ffmpeg.stream.camera_1] - FFmpeg decoder command: ffmpeg_camera_1 -hide_banner -loglevel error -avoid_negative_ts make_zero -fflags nobuffer -flags low_delay -strict experimental -fflags +genpts -use_wallclock_as_timestamps 1 -vsync 0 -timeout 5000000 -c:v h264 -rtsp_transport tcp -i rtsp://10.5.45.251:554/s0 -f segment -segment_time 5 -reset_timestamps 1 -strftime 1 -c:v copy -c:a aac /segments/camera_1/%Y%m%d%H%M%S.mp4 -vf fps=1.0 -f rawvideo -pix_fmt nv12 pipe:1
[2023-10-23 14:30:51] [INFO ] [viseron.components.nvr.nvr.camera_1] - NVR for camera MyCam1 initialized
[2023-10-23 14:30:51] [INFO ] [viseron.components] - Setup of domain nvr for component nvr with identifier camera_1 took 0.0 seconds
[2023-10-23 14:30:51] [INFO ] [viseron.core] - Viseron initialized in 59.8 seconds
[2023-10-23 14:30:54] [DEBUG ] [viseron.components.nvr.nvr.camera_1] - First frame received
[2023-10-23 14:30:54] [DEBUG ] [viseron.components.nvr.nvr.camera_1] - Starting object detector
[2023-10-23 14:30:55] [DEBUG ] [viseron.components.edgetpu.object_detector.camera_1] - Objects: []
[2023-10-23 14:30:55] [DEBUG ] [viseron.components.nvr.nvr.camera_1] - Not recording, pausing object detector
F driver/usb/usb_driver.cc:1148] HandleQueuedBulkIn transfer in failed. Not found: USB transfer error 5 [LibUsbDataInCallback]
Ahh now i think i know whats up. The problem is that you are using both the image classifier and the object detector on the same coral.
Since you havent specified a device for the object detector, it will default to using the first Coral device it can find, which clashes with the image classifier.
Can you try this?
edgetpu:
object_detector:
device: cpu
cameras:
camera_1:
fps: 1
labels:
- label: person
confidence: 0.8
image_classification:
# Use any USB Edge TPU
device: usb
cameras:
camera_1:
labels:
- person
Thank you, this way no error is shown. I took the configuration from the components example and changed cpu
to usb
at image_classification. Seems I misunderstood that the edgetpu can be used for both object_detector and image_classification. Ok, so I guess I rather use the default as given. But why would one need image_classification anyways while object_detector is already working on the cam?
Image classification works in a different way. It's a post processor that runs on detected objects. For instance, you can setup the object detector to detect birds, and then pass those along to the image classifier which could determine what type of bird it is.
There is some more information on this here: https://viseron.netlify.app/docs/documentation/configuration#image-classification-domain
Ah ok, thank you for the example and clarification. Are there some recommendations what requires more ressources or what is very cpu intense and where an edgetpu is better suited to be used? Or is it better to have several edgetpus to use an edgetpu for every component?
Definitely for object detection, as it runs more often in normal cases.
Depending on your hardware you could look into the darknet
component which has other means of hardware acceleration without the edgetpu
Ok, got it, thank you very much for the explanation!
I am trying to use the coral.ai USB Accelerator within a Proxmox VM using docker command:
sudo docker run --privileged --device /dev/bus/usb:/dev/bus/usb --rm -v /var/opt/recordings:/recordings -v /etc/viseron:/config -v /etc/localtime:/etc/localtime:ro -p 8888:8888 --name viseron roflcoopter/viseron:latest
Also I tried:
But the edgetpu won't be utilized, altough it gets recognized. What could I check/do to get it working?
The accelerator is availble in the VM and an example test works fine:
config:
Following error(s) in the log: