search

luxonis / depthai-experiments

Experimental projects we've done with DepthAI.
MIT License
801 stars 357 forks source link

Collision avoidance not working #459

Closed bruno-darochac closed 1 year ago

bruno-darochac commented 1 year ago

Hello ! I would like to test your collision avoidance. I know that it is always on a gen1 implementation. I see in an other issue that just installing the requirements should make the code work, but I see no spcification of version in the requirements :

imutils
concurrent-log-handler
opencv-python
scipy
--extra-index-url https://artifacts.luxonis.com/artifactory/luxonis-python-snapshot-local/
depthai

Here is the error message i got : 

TypeError: __init__(): incompatible constructor arguments. The following argument types are supported:
    1. depthai.Device(pipeline: depthai.Pipeline)
    2. depthai.Device(pipeline: depthai.Pipeline, usb2Mode: bool)
    3. depthai.Device(pipeline: depthai.Pipeline, maxUsbSpeed: depthai.UsbSpeed)
    4. depthai.Device(pipeline: depthai.Pipeline, pathToCmd: Path)
    5. depthai.Device(pipeline: depthai.Pipeline, devInfo: depthai.DeviceInfo, usb2Mode: bool = False)
    6. depthai.Device(pipeline: depthai.Pipeline, deviceInfo: depthai.DeviceInfo, maxUsbSpeed: depthai.UsbSpeed)
    7. depthai.Device(pipeline: depthai.Pipeline, devInfo: depthai.DeviceInfo, pathToCmd: Path)
    8. depthai.Device(version: depthai.OpenVINO.Version = <Version.???: 7>)
    9. depthai.Device(version: depthai.OpenVINO.Version, usb2Mode: bool = False)
    10. depthai.Device(version: depthai.OpenVINO.Version, maxUsbSpeed: depthai.UsbSpeed)
    11. depthai.Device(version: depthai.OpenVINO.Version, pathToCmd: Path)
    12. depthai.Device(version: depthai.OpenVINO.Version, deviceInfo: depthai.DeviceInfo, usb2Mode: bool = False)
    13. depthai.Device(version: depthai.OpenVINO.Version, deviceInfo: depthai.DeviceInfo, maxUsbSpeed: depthai.UsbSpeed)
    14. depthai.Device(version: depthai.OpenVINO.Version, deviceDesc: depthai.DeviceInfo, pathToCmd: Path)
    15. depthai.Device(config: depthai.Device.Config)
    16. depthai.Device(config: depthai.Device.Config, deviceInfo: depthai.DeviceInfo)
    17. depthai.Device(deviceInfo: depthai.DeviceInfo)
    18. depthai.Device(deviceInfo: depthai.DeviceInfo, maxUsbSpeed: depthai.UsbSpeed)
    19. depthai.Device(nameOrDeviceId: str)
    20. depthai.Device(nameOrDeviceId: str, maxUsbSpeed: depthai.UsbSpeed)

Invoked with: '', False
Exception ignored in: <function DepthAI.__del__ at 0x7f759d5275b0>

It appear when trying to initialise the device on depthai_utils.py

Is it always runnable ? And if yes is it fixable ?

Thanks in advance for your answer

Erol444 commented 1 year ago

Hi @bruno-darochac , We apologize for the incorrect requirements, it should be depthai==1.0.0.0. Could you try installing that first? Thanks, Erik

bruno-darochac commented 1 year ago

Hello thanks for your fast answer !

It works for the installation and the running, but know when your program draw a rectangle on the image it crashed with this output

Traceback (most recent call last):
  File "/home/bruno/dev/FIAT/RIP_CSEM/depthai-experiments/collision-avoidance/main.py", line 121, in <module>
    MainDebug().run()
  File "/home/bruno/dev/FIAT/RIP_CSEM/depthai-experiments/collision-avoidance/main.py", line 34, in run
    self.parse_frame(frame, results)
  File "/home/bruno/dev/FIAT/RIP_CSEM/depthai-experiments/collision-avoidance/main.py", line 92, in parse_frame
    super().parse_frame(frame, results)
  File "/home/bruno/dev/FIAT/RIP_CSEM/depthai-experiments/collision-avoidance/main.py", line 26, in parse_frame
    crash_alert = self.crash_avoidance.parse(tracker_objs)
  File "/home/bruno/dev/FIAT/RIP_CSEM/depthai-experiments/collision-avoidance/crash_avoidance.py", line 81, in parse
    if len(self.entries[key]) > 2 and self.is_dangerous_trajectory(key) and self.is_impact_close(key):
  File "/home/bruno/dev/FIAT/RIP_CSEM/depthai-experiments/collision-avoidance/crash_avoidance.py", line 29, in is_dangerous_trajectory
    m, b = self.best_fit_slope_and_intercept(objectID)
  File "/home/bruno/dev/FIAT/RIP_CSEM/depthai-experiments/collision-avoidance/crash_avoidance.py", line 23, in best_fit_slope_and_intercept
    m, b = np.polyfit(xs, zs, 1)
  File "<__array_function__ internals>", line 200, in polyfit
  File "/home/bruno/dev/FIAT/RIP_CSEM/depthai-experiments/collision-avoidance/venv/lib/python3.10/site-packages/numpy/lib/polynomial.py", line 668, in polyfit
    c, resids, rank, s = lstsq(lhs, rhs, rcond)
  File "<__array_function__ internals>", line 200, in lstsq
  File "/home/bruno/dev/FIAT/RIP_CSEM/depthai-experiments/collision-avoidance/venv/lib/python3.10/site-packages/numpy/linalg/linalg.py", line 2285, in lstsq
    x, resids, rank, s = gufunc(a, b, rcond, signature=signature, extobj=extobj)
  File "/home/bruno/dev/FIAT/RIP_CSEM/depthai-experiments/collision-avoidance/venv/lib/python3.10/site-packages/numpy/linalg/linalg.py", line 101, in _raise_linalgerror_lstsq
    raise LinAlgError("SVD did not converge in Linear Least Squares")
numpy.linalg.LinAlgError: SVD did not converge in Linear Least Squares

Here are the version of opencv and numpy that I have on my venv numpy==1.24.2 opencv-python==4.7.0.72

Should I have other one ? Thanks in advance

Erol444 commented 1 year ago

Hi @bruno-darochac , I'd suggest using these requirements, and for numpy I would take the latest that was released back then (2 years ago). Thoughts? Thanks, Erik

bruno-darochac commented 1 year ago

Hello, this allow me to launch the program.

But I have to comportement now,

  1. When I start the programm and have my hand directly in front of it, it surround my hand with a green boxes and crash. Output is : 
    
/home/bruno/dev/FIAT/RIP_CSEM/depthai-experiments/collision-avoidance/venv/lib/python3.10/site-packages/numpy/lib/polynomial.py:667: RuntimeWarning: invalid value encountered in divide
lhs /= scale
** On entry to DLASCL parameter number  4 had an illegal value
** On entry to DLASCL parameter number  4 had an illegal value
** On entry to DLASCL parameter number  4 had an illegal value
** On entry to DLASCL parameter number  4 had an illegal value
** On entry to DLASCL parameter number  5 had an illegal value
** On entry to DLASCL parameter number  4 had an illegal value
=== New data in observer stream host_capture, size: 4
Writing 4 bytes to host_capture
!!! XLink write successful: host_capture (4)
Stopping threads: ...
E: [global] [    843163] [EventRead00Thr] handleIncomingEvent:563   handleIncomingEvent() Read failed -4

E: [global] [ 843164] [python3] addEvent:262 Condition failed: event->header.flags.bitField.ack != 1 E: [global] [ 843164] [python3] addEventWithPerf:276 addEvent(event) method call failed with an error: 3 E: [global] [ 843164] [python3] XLinkReadData:156 Condition failed: (addEventWithPerf(&event, &opTime)) Device get data failed: 7 Closing stream metaout: ... E: [global] [ 843164] [python3] addEvent:262 Condition failed: event->header.flags.bitField.ack != 1 E: [global] [ 843164] [python3] addEventWithPerf:276 addEvent(event) method call failed with an error: 3 E: [global] [ 843164] [python3] XLinkReadData:156 Condition failed: (addEventWithPerf(&event, &opTime)) Device get data failed: 7 Closing stream previewout: ... E: [global] [ 843164] [Scheduler00Thr] dispatcherEventSend:53 Write failed (header) (err -4) | event XLINK_CLOSE_STREAM_REQ

Closing stream metaout: DONE. Thread for metaout finished. E: [xLink] [ 843164] [Scheduler00Thr] sendEvents:1027 Event sending failed E: [global] [ 843164] [Scheduler00Thr] dispatcherEventSend:53 Write failed (header) (err -4) | event XLINK_CLOSE_STREAM_REQ

E: [xLink] [ 843164] [Scheduler00Thr] sendEvents:1027 Event sending failed Closing stream previewout: DONE. Thread for previewout finished. Stopping threads: DONE 0.042s. Closing all observer streams: ... Closing stream host_capture: ... Closing stream host_capture: DONE. Closing all observer streams: DONE. Reseting device: 0. E: [global] [ 843164] [Scheduler00Thr] dispatcherEventSend:53 Write failed (header) (err -4) | event XLINK_RESET_REQ

E: [xLink] [ 843164] [Scheduler00Thr] sendEvents:1027 Event sending failed E: [xLink] [ 843165] [python3] DispatcherWaitEventComplete:367 Assertion Failed: curr != NULL

E: [global] [ 843165] [python3] XLinkResetRemote:254 Condition failed: DispatcherWaitEventComplete(&link->deviceHandle) Reseting: DONE. 2023-03-07 10:37:23,165 - depthai_utils - INFO - [INFO] elapsed time: 2.19 2023-03-07 10:37:23,173 - depthai_utils - INFO - [INFO] approx. FPS: 24.64 Traceback (most recent call last): File "/home/bruno/dev/FIAT/RIP_CSEM/depthai-experiments/collision-avoidance/main.py", line 121, in MainDebug().run() File "/home/bruno/dev/FIAT/RIP_CSEM/depthai-experiments/collision-avoidance/main.py", line 34, in run self.parse_frame(frame, results) File "/home/bruno/dev/FIAT/RIP_CSEM/depthai-experiments/collision-avoidance/main.py", line 92, in parse_frame super().parse_frame(frame, results) File "/home/bruno/dev/FIAT/RIP_CSEM/depthai-experiments/collision-avoidance/main.py", line 26, in parse_frame crash_alert = self.crash_avoidance.parse(tracker_objs) File "/home/bruno/dev/FIAT/RIP_CSEM/depthai-experiments/collision-avoidance/crash_avoidance.py", line 81, in parse if len(self.entries[key]) > 2 and self.is_dangerous_trajectory(key) and self.is_impact_close(key): File "/home/bruno/dev/FIAT/RIP_CSEM/depthai-experiments/collision-avoidance/crash_avoidance.py", line 29, in is_dangerous_trajectory m, b = self.best_fit_slope_and_intercept(objectID) File "/home/bruno/dev/FIAT/RIP_CSEM/depthai-experiments/collision-avoidance/crash_avoidance.py", line 23, in best_fit_slope_and_intercept m, b = np.polyfit(xs, zs, 1) File "<__array_function__ internals>", line 200, in polyfit File "/home/bruno/dev/FIAT/RIP_CSEM/depthai-experiments/collision-avoidance/venv/lib/python3.10/site-packages/numpy/lib/polynomial.py", line 668, in polyfit c, resids, rank, s = lstsq(lhs, rhs, rcond) File "<__array_function__ internals>", line 200, in lstsq File "/home/bruno/dev/FIAT/RIP_CSEM/depthai-experiments/collision-avoidance/venv/lib/python3.10/site-packages/numpy/linalg/linalg.py", line 2285, in lstsq x, resids, rank, s = gufunc(a, b, rcond, signature=signature, extobj=extobj) File "/home/bruno/dev/FIAT/RIP_CSEM/depthai-experiments/collision-avoidance/venv/lib/python3.10/site-packages/numpy/linalg/linalg.py", line 101, in _raise_linalgerror_lstsq raise LinAlgError("SVD did not converge in Linear Least Squares") numpy.linalg.LinAlgError: SVD did not converge in Linear Least Squares



2. And the second is that if nothing is in front it will crash the same way as before when someting pass in front of the camera..
Erol444 commented 1 year ago

Hi @bruno-darochac , This API and demo have been deprecated for 2 years now, and we will not be fixing them. For collision avoidance, I would suggest using ROS, where such tasks are already "solved". Thanks, Erik

michaelnguyen11 commented 1 year ago

Hi @Erol444 ,

Thank you for your suggestion. I'm curious that the ROS you mention is to use ROS standard API, or use depthai-ros repository ? I can't found a collision avoidance demo in depthai-ros repository, can you point out where I can find it in Luxonis's repositories.

Thanks in advance !

Br, Michael

Erol444 commented 1 year ago

Hi @michaelnguyen11 , I was referring to ROS itself, I believe there should be quite a few opensource projects/demos doing collision avoidance inside ros (based on pointclouds). Thanks, Erik