search

snowzach / doods2

API for detecting objects in images and video streams using Tensorflow
MIT License
221 stars 28 forks source link

Could not create detector, RPi4 (deepstack, pytorch) #96

Closed igneon-ha closed 5 months ago

igneon-ha commented 8 months ago

Greetings!

I'm having a problem that, based on my research here and on other resources, has already occurred earlier.

Having pulled and run "latest" image on RPi4 I see the following errors in the container log. Needless to say that all the attempts to use both pytorch and deepstack models lead to failure.

Could you, please, help with the situation?

Basic built-in config used for starters:

root@3cdd1427f1f0:/opt/doods# cat config.yaml
doods:
  boxes:
    enabled: True
    boxColor: [0, 255, 0]
    boxThickness: 1
    fontScale: 1.2
    fontColor: [0, 255, 0]
    fontThickness: 1
  regions:
    enabled: True
    boxColor: [255, 0, 255]
    boxThickness: 1
    fontScale: 1.2
    fontColor: [255, 0, 255]
    fontThickness: 1
  globals:
    enabled: True
    fontScale: 1.2
    fontColor: [255, 255, 0]
    fontThickness: 1
  detectors:
    - name: default
      type: tflite
      modelFile: models/coco_ssd_mobilenet_v1_1.0_quant.tflite
      labelFile: models/coco_labels0.txt
    - name: tensorflow
      type: tensorflow
      modelFile: models/faster_rcnn_inception_v2_coco_2018_01_28.pb
      labelFile: models/coco_labels1.txt
    - name: pytorch
      type: pytorch
      modelFile: ultralytics/yolov5,yolov5s

Logs with basic config:

root@ubuntu:~# docker logs -f 3cdd1427f1f0 2023-11-03 19:13:56,531 - doods.doods - INFO - Registered detector type:tflite name:default 2023-11-03 19:14:03,195 - doods.doods - INFO - Registered detector type:tensorflow name:tensorflow 2023-11-03 19:14:03,195 - doods.doods - ERROR - Could not create detector pytorch/pytorch: Unknown detector type pytorch. 2023-11-03 19:14:03,317 - uvicorn.error - INFO - Started server process [1] 2023-11-03 19:14:03,318 - uvicorn.error - INFO - Waiting for application startup. 2023-11-03 19:14:03,319 - uvicorn.error - INFO - Application startup complete. 2023-11-03 19:14:03,320 - uvicorn.error - INFO - Uvicorn running on http://0.0.0.0:8080

Config with added deepstack directive:

GNU nano 4.8                                                                              config.yaml                                                                               Modified
doods:
  boxes:
    enabled: True
    boxColor: [0, 255, 0]
    boxThickness: 1
    fontScale: 1.2
    fontColor: [0, 255, 0]
    fontThickness: 1
  regions:
    enabled: True
    boxColor: [255, 0, 255]
    boxThickness: 1
    fontScale: 1.2
    fontColor: [255, 0, 255]
    fontThickness: 1
  globals:
    enabled: True
    fontScale: 1.2
    fontColor: [255, 255, 0]
    fontThickness: 1
  detectors:
    - name: default
      type: tflite
      modelFile: models/coco_ssd_mobilenet_v1_1.0_quant.tflite
      labelFile: models/coco_labels0.txt
    - name: tensorflow
      type: tensorflow
      modelFile: models/faster_rcnn_inception_v2_coco_2018_01_28.pb
      labelFile: models/coco_labels1.txt
    - name: pytorch
      type: pytorch
      modelFile: ultralytics/yolov5,yolov5s
    - name: dark
      type: deepstack
      modelFile: models/dark.pt

The log is as follows:

root@ubuntu:~# docker logs -f 3cdd1427f1f0 2023-11-03 19:13:56,531 - doods.doods - INFO - Registered detector type:tflite name:default 2023-11-03 19:14:03,195 - doods.doods - INFO - Registered detector type:tensorflow name:tensorflow 2023-11-03 19:14:03,195 - doods.doods - ERROR - Could not create detector pytorch/pytorch: Unknown detector type pytorch. 2023-11-03 19:14:03,317 - uvicorn.error - INFO - Started server process [1] 2023-11-03 19:14:03,318 - uvicorn.error - INFO - Waiting for application startup. 2023-11-03 19:14:03,319 - uvicorn.error - INFO - Application startup complete. 2023-11-03 19:14:03,320 - uvicorn.error - INFO - Uvicorn running on http://0.0.0.0:8080 (Press CTRL+C to quit) 2023-11-03 19:34:20,567 - uvicorn.error - INFO - Shutting down 2023-11-03 19:34:20,668 - uvicorn.error - INFO - Waiting for application shutdown. 2023-11-03 19:34:20,670 - uvicorn.error - INFO - Application shutdown complete. 2023-11-03 19:34:20,670 - uvicorn.error - INFO - Finished server process [1] 2023-11-03 19:34:29,995 - doods.doods - INFO - Registered detector type:tflite name:default 2023-11-03 19:34:36,537 - doods.doods - INFO - Registered detector type:tensorflow name:tensorflow 2023-11-03 19:34:36,537 - doods.doods - ERROR - Could not create detector deepstack/dark: Unknown detector type deepstack. 2023-11-03 19:34:36,537 - doods.doods - ERROR - Could not create detector pytorch/pytorch: Unknown detector type pytorch. 2023-11-03 19:34:36,655 - uvicorn.error - INFO - Started server process [1] 2023-11-03 19:34:36,656 - uvicorn.error - INFO - Waiting for application startup. 2023-11-03 19:34:36,657 - uvicorn.error - INFO - Application startup complete. 2023-11-03 19:34:36,658 - uvicorn.error - INFO - Uvicorn running on http://0.0.0.0:8080

Tried to run it with no directives but the deepstack one - same result.

hkrob commented 6 months ago

Same here on the odroid-N2+

stormshaker commented 5 months ago

I have the same issue on the same platform as you (Pi4). Mine is HA OS. Same error 'Could not create detector pytorch/pytorch' when using the default config.

hkrob commented 5 months ago

Turns out pytorch is not supported on pi4 It's in the docs... I missed it

On Mon, Jan 22, 2024, 16:48 James Nobes @.***> wrote:

I have the same issue on the same platform as you (Pi4). Mine is HA OS. Same error 'Could not create detector pytorch/pytorch' when using the default config.

— Reply to this email directly, view it on GitHub https://github.com/snowzach/doods2/issues/96#issuecomment-1903511117, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACSU3WH4YKAC2A46HPXMRUTYPYRUNAVCNFSM6AAAAAA6445U4GVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSMBTGUYTCMJRG4 . You are receiving this because you commented.Message ID: @.***>

stormshaker commented 5 months ago

Tested on an amd64 and pytorch downloads requirements and works. Looks like it can be compiled for the pi4, but it's painful: https://qengineering.eu/install-pytorch-on-raspberry-pi-4.html

igneon-ha commented 5 months ago

Well, it sucks :( Guys, thank you for the tips, will go look for an amd64 device then.

hkrob commented 5 months ago

Maybe an n100 based mini?

On Tue, Jan 23, 2024, 02:44 igneon-ha @.***> wrote:

Well, it sucks :( Guys, thank you for the tips, will go look for an amd64 device then.

— Reply to this email directly, view it on GitHub https://github.com/snowzach/doods2/issues/96#issuecomment-1904595313, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACSU3WBWKL5BULBMDCHVPYTYP2XQ3AVCNFSM6AAAAAA6445U4GVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSMBUGU4TKMZRGM . You are receiving this because you commented.Message ID: @.***>

igneon-ha commented 5 months ago

@hkrob That's right, this is the option I've thought of, though I have to keep energy efficiency in mind, as there are rare power outages in that place (critical stuff is on UPSes), and an n100-based mini is unlikely to reach the rpi4 level. But alas.

stormshaker commented 5 months ago

I tried all the Google tflite models last night on a USB coral device on the pi4 and they were fast, but had awful accuracy. I ended up selecting the tensorflow detector, even though it takes about 5 seconds to run. I just scheduled it to only run when motion is detected. I'm also hoping when the pi5 is fully supported by HA, the tensorflow model will run faster.

hkrob commented 5 months ago

e

Right.. but I think the flexibility/support for the x86-64 platform is going to outweigh the small power increase My next build will probably be n100 based with Frigate + Doods + Coral ...

igneon-ha commented 5 months ago

@stormshaker Thanks for your observations, never thought the coral thing would be so inefficient precision-wise. Waiting for RPi5 is a good point, I assume.

@hkrob Sure, your reasoning definitely makes sense and I'll give one of the n100-based platforms a try. Though I'd say that opting for 5 second processing time on PRi4 and only when there's a motion detection present, as @stormshaker mentioned, is also a tolerable trade-off.

Do you guys use your setups in your home surveillance system? I'm wondering cause I've tied to do just that and faced the issue of frustratingly poor detection precision at night with literally any "standard" model. Then I thought that the DeepStack dark.pt model would make a difference but have never made it to check the hypothesis due to the issue in this post. And since training my own model now seems not to be an option because of the complexity of the process, I feel I'm stuck with this. Do you have any thoughts on the situation?

stormshaker commented 5 months ago

For me, the out of the box model included with doods2 and the tensorflow detector works even at night. My CCTV camera has a view of the driveway, and it detects when a car is present or not. Sometimes it gets two cars instead of one, but that's a minor problem. Any option for you to move your camera or add better infrared lighting?

snowzach commented 5 months ago

FWIW, if you install the 64 bit OS on the Raspberry Pi4 you can run the AARCH/ARM64 version of doods which includes support for tensorflow as well as tensorflow lite but I have found it still doesn't really have enough power to run pytorch. I may enable it at some point. I finally gave up supporting the 32 bit arm7 images. They are just too old to function.

igneon-ha commented 5 months ago

@stormshaker Good to know that standard models satisfy you, at least regarding car detection.

I've been trying to figure out how it all works for a bit. As I understand it, models trained on color images, in principle, cannot work normally with BW, because such images have two color channels instead of three, which breaks the logic of the model. Apparently, it is for this reason that in my case the corner of the woodshed is defined as a person, one of the bushes as a toilet and so on, and even artificial illumination of these objects with a flashlight doesn't help.

So, I'll try to experiment with non-standard models or try to calorize images with AI (haha).

@snowzach Thanks for the insight, Zach. RPi4 is now written off for this task :)