OAK-D Lite

[rexn8r]

Hi

I am trying to setup CAIT on my raspberry pi model 4 8 GB with 64 bit OS.

Ran the setup.sh to install pre-requisites.

Now when i am trying to run the setup; the page is unable to detect OAK-D camera. I have installed DepthAI pre-requisites and ran Age & Gender demo and its running successfully.

Below is the setup page with Devtools;

here's the OAK-D camera which is functioning well;

below is the lsusb command result;

how do i complete the setup?

thanks rex

[gris74]

Yeah same problem here when fresh install on a pi4 with a OAK-D-Lite ...

[michaelhwn]

Hi @rexn8r and @gris74, I will be looking into this problem. I will let you know as soon as I figured out the cause of this issue.

[rexn8r]

Thanks @michaelhwn

looking forward to the solution.

One thing I missed to mention is that while running setup.sh, the opencv-contrib-python installation got stuck with following message;

Building wheel for opencv-contrib-python (PEP 517)....... /

I then cancelled the process after one hour, manually installed opencv-contrib-python and ran the setup again.

thanks rex

[rexn8r]

Hi @michaelhwn

Any update on the resolution for the issues i faced above?

Thanks

[michaelhwn]

Hi @rexn8r, I am trying to reproduce it on my end. One thing I noticed is that the long installation time of openCV only happens on a 64-bit Raspberry Pi OS (Bullseye). Is this the OS you are using?

[michaelhwn]

Hi @rexn8r and @gris74, so far I can reproduce this issue on the latest 64-bit Raspberry Pi OS (Bullseye). The reason is that CEP's vision module hasn't been updated to support 64-bit OS. This caused the module to fail to initialize. On a 32-bit system, I haven't been able to reproduce this error. If you are using a 32-bit OS and encounter this error, please run the following commands and provide the output messages to me:

sudo journalctl -u cait_webapp.service -f sudo journalctl -u cait_webapp.service -f

Regardless, I will be updating CEP's vision module to support 64-bit OS in a day or two. I will also let you know once the updates are pushed.

[rexn8r]

hi @michaelhwn

I have checked the pi version and its 32 bit.

below is the output message of the service;

+++++++++++++++++++++++ Mar 14 10:52:58 raspberrypi bash[422]: rv = self.dispatch_request() Mar 14 10:52:58 raspberrypi bash[422]: File "/usr/lib/python3/dist-packages/flask/app.py", line 1936, in dispatch_request Mar 14 10:52:58 raspberrypi bash[422]: return self.view_functionsrule.endpoint Mar 14 10:52:58 raspberrypi bash[422]: File "/home/pi/cep/src/cait/cortic_webapp/main.py", line 282, in getvideodev Mar 14 10:52:58 raspberrypi bash[422]: devices = essentials.get_video_devices() Mar 14 10:52:58 raspberrypi bash[422]: NameError: name 'essentials' is not defined Mar 14 10:52:58 raspberrypi bash[422]: 192.168.1.3 - - [14/Mar/2022 10:52:58] "GET /getvideodev HTTP/1.1" 500 - Mar 14 10:52:58 raspberrypi bash[422]: INFO:werkzeug:192.168.1.3 - - [14/Mar/2022 10:52:58] "GET /getvideodev HTTP/1.1" 500 - Mar 14 10:53:00 raspberrypi bash[422]: 192.168.1.3 - - [14/Mar/2022 10:53:00] "GET /getwifi HTTP/1.1" 200 - Mar 14 10:53:00 raspberrypi bash[422]: INFO:werkzeug:192.168.1.3 - - [14/Mar/2022 10:53:00] "GET /getwifi HTTP/1.1" 200 -

+++++++++++++++++++++++++++++++++++++

thanks rex

[michaelhwn]

Thanks for the log @rexn8r, the error shows that the “essential” module of CAIT is not found, and thus the calling of any functions provided by that module returns an error. This led me to suspect that something went wrong during the setup process. Specifically, the process might have stopped before this line. It is possible that the long installation time of opencv-contrib-python (because it is probably compiling from scratch) caused such issue. I will be testing with other openCV versions and find one that can be installed much faster.

In the meantime, can you try to setup CEP on a newly flashed 32-bit Raspberry Pi OS again? But this time, just let the opencv-contrib-python installation run until the setup process finish and reboot the Pi.

[rexn8r]

sure thing. i am onto it. will update you how it goes.

thanks rex

[rexn8r]

hi @michaelhwn

finally managed to install and setup on raspberry pi 4. this time the installation completed quite fast and without error and also the opencv wheel didn't take long..

2 questions about the solution:

1. How do i add age & gender or other neural network in the programming section?
2. can i setup and use CAIT on windows PC?

thanks rex

[michaelhwn]

Hi @rexn8r, good to know it works this time. In order to add a new vision capability, you have to perform the following steps:

Note: I will be using the OAK-D Facemesh function as an example since adding age & gender will be a very similar process

1. In <root_of_cep>/src/curt/curt/modules/vision, you will need to create a new Python module that inherits the OAKDProcessingWorker class. An example can be found in here.
2. Similar to the example shown in 1., you will need to implement four functions: I. config_worker (You can leave it empty if your worker has nothing to configure) II. preprocess_input III. execute_nn_operation IV. postprocess_result
3. Add the age&gender worker as a new node type in <root_of_cep>/src/curt/curt/modules/vision/oakd_node_types.py. Similar to this example
4. In <root_of_cep>/src/curt/curt/module_configs.json, add a new key-value pair entry under vision, the key is the module name (filename without extension) you created in 1., and the value is the class name. Example. Then also add the module name into the worker_list. Example.
5. Add your age & gender model file into <root_of_cep>/http://src/curt/models/modules/vision/platforms/oakd/.
6. If you reboot the Pi at this stage, you can already use the age & gender function in the CURT programming interface similar to the example shown in here. If you also want to extend the CAIT visual programming interface with a new age & gender block, then please continue the following steps:
7. In <root_of_cep>/src/cait/cait/core.py, add a function that uses the CURT programming interface to involve the age & gender function. One example can be found in here.
8. In <root_of_cep>/src/cait/cait/essentials.py, add a function that calls the age&gender function you implemented in 6. All of the functions in this module form the CAIT Python programming interface. You can find an example in here.
9. In <root_of_cep>/src/cait/cortic_webapp/main.py. Implement a handler for the age & gender request. You can find an example in here.
10. In <root_of_cep>/src/cait/cortic_webapp/static/js/cait_blocks.js. Add a new block type for the age & gender block. This is an example for the facemesh block. For the block name and description, you need to extend the string table in <root_of_cep>/src/cait/cortic_webapp/static/js/en.js. Example.
11. Implement the Javascript and Python code generation process of this block. This is the example for Javascript, and this is the example for Python.
12. Extend the Javascript and Python code for oakd_pipeline block to include the new age & gender node. This is the example for Javascript, and this is the example for Python.
13. Add the Javascript function you created in 10 to vision_func like this.
14. Add the programing blocks that your new block depends on into the vision_func_dependent_blocks like this.
15. In <root_of_cep>/src/cait/cortic_webapp/static/js/cait_functions.js, implement the Javascript function you created in 10. Example.
16. In <root_of_cep>/src/cait/cortic_webapp/templates/programming.html, enable the age & gender block like this.
17. Up to this point, you should have a function age & gender block that you can use in CAIT's visual programming interface. However, the visualization of the result is not implemented yet, so you won't be able to see the age & gender results in the video stream. The process of creating the drawing block of age & gender is essentially a variation of steps 7 to 16. Below are the examples of drawing the facemesh results:
18. Extend core.py: Load the drawing function, and use the drawing function.
19. Extend essentials.py: Add new drawing function.
20. Extend main.py: Add handler for drawing.
21. Extend cait_blocks.js: Add new drawing block, implement Javascript and Python code, and add to vision_func.
22. Enable the block in programming.html.

The above steps should give you a complete age & gender function in the CAIT visual programming interface, CAIT Python programming interface, and CURT programming interface. Please feel free to contact me if you encounter any issues in following the steps.

Regarding the Windows support of CAIT, currently, CAIT is only designed to run in the Raspberry Pi OS on a Raspberry 4B. So you won't be able to set it up on a Windows machine.

[rexn8r]

hi @michaelhwn

thanks for the guideline. I will give it a shot and let you know.

regards

[michaelhwn]

Closing due to inactivity.