search

cortictechnology / cep

CEP is a software platform designed for users that want to learn or rapidly prototype using standard A.I. components.
MIT License
167 stars 21 forks source link

Bug fix and improvements #3

Closed michaelhwn closed 3 years ago

michaelhwn commented 3 years ago

Bug fixed:

  1. When a depthAI pipeline is defined and started by the frontend (CAIT of CURT's Python command interface), CEP's backend, CURT will also start a thread to keep getting the node data from the depthAI pipeline. These node data can either be video frames or neural network inference results, and they all have an initial default value of "None" in the thread. There was an incorrect assumption that the thread can start as soon as the pipeline is started to obtain data immediately to replace the "None" values. This assumption worked with the OAK-D camera, and so there weren't issues observed. However, when testing with the OAK-D-Lite camera, after the pipeline is started, there is a short period that the thread is not able to obtain data from the pipeline. This is because of the depthai.DataOutputQueue.get() function, which is a blocking call, doesn't return in a short time after the pipeline is started. Therefore, during this waiting/blocking period, the node data in the thread are all "None". As a result, if there are requests to get these node data from the frontend during this period, only the "None" values will be returned. This situation was not handled and thus caused errors. This was fixed by adding proper handling of None type data in CEP's frontend.

Improvements:

  1. Added a checking mechanism in the Visual Programming Interface to ensure users have added the proper depthAI nodes before executing a program with the OAKD camera.
  2. Improved the speed of hand landmark estimation by reusing previous landmarks to replace palm detection in the next frame. This is modified from geaxgx's work (https://github.com/geaxgx/depthai_hand_tracker) to support 2-hand estimation.