Using Webcam instead of raspberry Pi camera.

[bojuelkjeldsen]

I have a well functioning webcam, which can take pictures in Cheese (Ubuntu 22,04) and record sound/words. However, I seem to have trouble using the webcam for the MMM-Face-Reco-DNN? In an previously version of MMM-Face-Reco-DNN, I could use the webcam for facial reconition.

[nischi]

Hi @bojuelkjeldsen

Do you have a little bit more information whats wrong and maybe error message you got?

Kind regards,

[bojuelkjeldsen]

Hi @nischi Thank you very much for your reply. I have had great use of your previous version of MMM-Face-Reco-DNN! My family likes the different versions made for each of them. In the previous version, you could set the "source" in confic.js to "1" if it was not a pi-cam. This option I cannot find in the new version? Regarding the error-messages, I am sorry, but I am not sure where to look for the error messages? The mirror runs fin, but no faces are recognized. The dataset and coding also runs without problems. While the mirror runs with the MMM-Face-Reco-DNN module, I can still access the webcam with Cheese, so I am not sure that webcam is active for the MMM-Face-Reco-DNN module? I am not sure that the module gets any pictures from the camera?

Best

Bo

[nischi]

Sorry i really dont know what could be wrong without some errors. If you run MM with pm2 you can check the error logs there.

https://pm2.keymetrics.io/docs/usage/log-management/

[bojuelkjeldsen]

Hi Nischi Thank you, I will check on the error logs! I do run the MM with PM2 as automatic startup.

[bojuelkjeldsen]

Hi Nischi Now, I checked the PM2 mm-error.log. It looks like the module "libcamera" is not installed for Python? (PythonShellError). I am not quite sure how to correct this? Reinstall OpenCV?

Best Bo

The Error log file:

[4587:0329/154802.262639:ERROR:gbm_wrapper.cc(253)] Failed to export buffer to dma_buf: No such file or directory (2) [4587:0329/154802.270248:ERROR:gbm_wrapper.cc(253)] Failed to export buffer to dma_buf: No such file or directory (2) [4587:0329/154802.271143:ERROR:gbm_wrapper.cc(253)] Failed to export buffer to dma_buf: No such file or directory (2) [4587:0329/154802.272030:ERROR:gbm_wrapper.cc(253)] Failed to export buffer to dma_buf: No such file or directory (2) [4587:0329/154802.272885:ERROR:gbm_wrapper.cc(253)] Failed to export buffer to dma_buf: No such file or directory (2) [4587:0329/154802.274088:ERROR:gbm_wrapper.cc(253)] Failed to export buffer to dma_buf: No such file or directory (2) [4587:0329/154802.274878:ERROR:gbm_wrapper.cc(253)] Failed to export buffer to dma_buf: No such file or directory (2) [4587:0329/154802.281378:ERROR:gbm_wrapper.cc(253)] Failed to export buffer to dma_buf: No such file or directory (2) [4587:0329/154802.282353:ERROR:gbm_wrapper.cc(253)] Failed to export buffer to dma_buf: No such file or directory (2) [4587:0329/154802.283281:ERROR:gbm_wrapper.cc(253)] Failed to export buffer to dma_buf: No such file or directory (2) [4587:0329/154802.284224:ERROR:gbm_wrapper.cc(253)] Failed to export buffer to dma_buf: No such file or directory (2) [4587:0329/154802.285183:ERROR:gbm_wrapper.cc(253)] Failed to export buffer to dma_buf: No such file or directory (2) [4587:0329/154802.286195:ERROR:gbm_wrapper.cc(253)] Failed to export buffer to dma_buf: No such file or directory (2) [4587:0329/154802.287095:ERROR:gbm_wrapper.cc(253)] Failed to export buffer to dma_buf: No such file or directory (2) [29.03.2024 15:48.20.641] [ERROR] Whoops! There was an uncaught exception... [29.03.2024 15:48.20.647] [ERROR] PythonShellError: ModuleNotFoundError: No module named 'libcamera' at PythonShell.parseError (/home/bo/MagicMirror/modules/MMM-Face-Reco-DNN/node_modules/python-shell/index.js:303:21) at terminateIfNeeded (/home/bo/MagicMirror/modules/MMM-Face-Reco-DNN/node_modules/python-shell/index.js:193:32) at ChildProcess. (/home/bo/MagicMirror/modules/MMM-Face-Reco-DNN/node_modules/python-shell/index.js:185:13) at ChildProcess.emit (node:events:514:28) at Process.onexit (node:internal/child_process:291:12) ----- Python Traceback ----- File "/home/bo/MagicMirror/modules/MMM-Face-Reco-DNN/tools/recognition.py", line 13, in from picamera2 import Picamera2 File "/home/bo/.local/lib/python3.10/site-packages/picamera2/init.py", line 3, in import libcamera { traceback: 'Traceback (most recent call last):\n' + ' File "/home/bo/MagicMirror/modules/MMM-Face-Reco-DNN/tools/recognition.py", line 13, in \n' + ' from picamera2 import Picamera2\n' + ' File "/home/bo/.local/lib/python3.10/site-packages/picamera2/init.py", line 3, in \n' + ' import libcamera\n' + "ModuleNotFoundError: No module named 'libcamera'\n", executable: 'python3', options: null, script: 'modules/MMM-Face-Reco-DNN/tools/recognition.py', args: [ '--cascade=modules/MMM-Face-Reco-DNN/model/haarcascade_frontalface_default.xml', '--encodings=modules/MMM-Face-Reco-DNN/model/encodings.pickle', '--rotateCamera=-1', '--method=dnn', '--detectionMethod=hog', '--interval=1500', '--output=0', '--outputmm=0', '--extendDataset=True', '--dataset=modules/MMM-Face-Reco-DNN/dataset/', '--tolerance=0.6', '--brightness=0', '--contrast=0', '--resolution=1920,1080', '--processWidth=500' ], exitCode: 1 } [29.03.2024 15:48.20.647] [ERROR] MagicMirror² will not quit, but it might be a good idea to check why this happened. Maybe no internet connection? [29.03.2024 15:48.20.649] [ERROR] If you think this really is an issue, please open an issue on GitHub: https://github.com/MichMich/MagicMirror/issues /home/bo/MagicMirror/node_modules/electron/dist/electron exited with signal SIGINT

[nischi]

i think you have not installed the picamera properly. try to reinstall all the dependencies

[bojuelkjeldsen]

Hi Nischi

Sorry for asking about this picamera2 and libcamera module. It seems to give some trouble. I have tried to reinstall these dependencies several times and in several ways. My system is ubuntu 22.04, 64 bit, Pi 4.
Should I somehow add a location for "libcamera" ? Perhaps try to add it to the path? Regarding the dependencies, I have tried to use npm ci, but that seems not to work on my Pi?

Best

Bo

bo@mm:~$ npm ci npm ERR! code ENOENT npm ERR! syscall open npm ERR! path /home/bo/package.json npm ERR! errno -2 npm ERR! enoent Could not read package.json: Error: ENOENT: no such file or directory, open '/home/bo/package.json' npm ERR! enoent This is related to npm not being able to find a file. npm ERR! enoent

The error.log from PM2 looks like this:

[31.03.2024 00:01.27.684] [ERROR] Whoops! There was an uncaught exception... [31.03.2024 00:01.27.708] [ERROR] PythonShellError: ImportError: cannot import name 'ControlType' from 'libcamera' (unknown location) at PythonShell.parseError (/home/bo/MagicMirror/modules/MMM-Face-Reco-DNN/node_modules/python-shell/index.js:303:21) at terminateIfNeeded (/home/bo/MagicMirror/modules/MMM-Face-Reco-DNN/node_modules/python-shell/index.js:193:32) at Socket. (/home/bo/MagicMirror/modules/MMM-Face-Reco-DNN/node_modules/python-shell/index.js:173:17) at Socket.emit (node:events:526:35) at endReadableNT (node:internal/streams/readable:1359:12) at process.processTicksAndRejections (node:internal/process/task_queues:82:21) ----- Python Traceback ----- File "/home/bo/MagicMirror/modules/MMM-Face-Reco-DNN/tools/recognition.py", line 13, in from picamera2 import picamera2 File "/home/bo/.local/lib/python3.10/site-packages/picamera2/picamera2/init.py", line 5, in from .configuration import CameraConfiguration, StreamConfiguration File "/home/bo/.local/lib/python3.10/site-packages/picamera2/picamera2/configuration.py", line 1, in from .controls import Controls File "/home/bo/.local/lib/python3.10/site-packages/picamera2/picamera2/controls.py", line 4, in from libcamera import ControlType, Rectangle, Size { traceback: 'Traceback (most recent call last):\n' + ' File "/home/bo/MagicMirror/modules/MMM-Face-Reco-DNN/tools/recognition.py", line 13, in \n' + ' from picamera2 import picamera2\n' + ' File "/home/bo/.local/lib/python3.10/site-packages/picamera2/picamera2/init.py", line 5, in \n' + ' from .configuration import CameraConfiguration, StreamConfiguration\n' + ' File "/home/bo/.local/lib/python3.10/site-packages/picamera2/picamera2/configuration.py", line 1, in \n' + ' from .controls import Controls\n' + ' File "/home/bo/.local/lib/python3.10/site-packages/picamera2/picamera2/controls.py", line 4, in \n' + ' from libcamera import ControlType, Rectangle, Size\n' + "ImportError: cannot import name 'ControlType' from 'libcamera' (unknown location)\n", executable: 'python3', options: null, script: 'modules/MMM-Face-Reco-DNN/tools/recognition.py', args: [ '--cascade=modules/MMM-Face-Reco-DNN/model/haarcascade_frontalface_default.xml', '--encodings=modules/MMM-Face-Reco-DNN/model/encodings.pickle', '--rotateCamera=-1', '--method=dnn', '--detectionMethod=hog', '--interval=1500', '--output=0', '--outputmm=0', '--extendDataset=True', '--dataset=modules/MMM-Face-Reco-DNN/dataset/', '--tolerance=0.6', '--brightness=0', '--contrast=0', '--resolution=1920,1080', '--processWidth=500' ], exitCode: 1

[bojuelkjeldsen]

[ERROR] PythonShellError: ModuleNotFoundError: No module named 'libcamera' at PythonShell.parseError (/home/bo/MagicMirror/modules/MMM-Face-Reco-DNN/node_modules/python-shell/index.js:303:21) at terminateIfNeeded (/home/bo/MagicMirror/modules/MMM-Face-Reco-DNN/node_modules/python-shell/index.js:193:32) at ChildProcess. (/home/bo/MagicMirror/modules/MMM-Face-Reco-DNN/node_modules/python-shell/index.js:185:13) at ChildProcess.emit (node:events:514:28) at Process.onexit (node:internal/child_process:291:12) ----- Python Traceback ----- File "/home/bo/MagicMirror/modules/MMM-Face-Reco-DNN/tools/recognition.py", line 13, in from picamera2 import Picamera2 File "/home/bo/.local/lib/python3.10/site-packages/picamera2/init.py", line 3, in import libcamera { traceback: 'Traceback (most recent call last):\n' + ' File "/home/bo/MagicMirror/modules/MMM-Face-Reco-DNN/tools/recognition.py", line 13, in \n' + ' from picamera2 import Picamera2\n' + ' File "/home/bo/.local/lib/python3.10/site-packages/picamera2/init.py", line 3, in \n' + ' import libcamera\n' + "ModuleNotFoundError: No module named 'libcamera'\n", executable: 'python3', options: null, script: 'modules/MMM-Face-Reco-DNN/tools/recognition.py', args: [ '--cascade=modules/MMM-Face-Reco-DNN/model/haarcascade_frontalface_default.xml', '--encodings=modules/MMM-Face-Reco-DNN/model/encodings.pickle', '--rotateCamera=-1', '--method=dnn', '--detectionMethod=hog', '--interval=1500', '--output=0', '--outputmm=0', '--extendDataset=True', '--dataset=modules/MMM-Face-Reco-DNN/dataset/', '--tolerance=0.6', '--brightness=0', '--contrast=0', '--resolution=1920,1080', '--processWidth=500' ], exitCode: 1

[bojuelkjeldsen]

Hi Nischi I do hope that your weekend has treated you well! It does seem that the libcamera module cannot be found. I have tried several ways to install it. When I manage to install it, it ends up installed in the root and not under the ".local/lib/python3.10/site-packages" which is where the picamera2 module is installed. The libcamera path has been added to the pythonpath. Unfortunately, this does not help? I am not sure how to install all the dependencies for the module in the right way? It runs on Raspberry pi 4, ubuntu 22.04, python 3.10, Suggestions are very much appriciated! Kind Regards Bo

[nischi]

Hi @bojuelkjeldsen

It seems you try to run npm ci in your home directory and not in the directory of the module itself.

And about the python dependencies. I'm not sure how it works with ubuntu, but could be that this is working same as on bookworm. Maybe you need to create a new virtual environment and install the dependencies there. Like described here: https://github.com/nischi/MMM-Face-Reco-DNN?tab=readme-ov-file#some-additional-steps-for-bookworm-and-above-to-run-it-with-an-virtual-environment

[bojuelkjeldsen]

Hi Nischi

Yes, thank you, it did help running the "npm ci" in the right directory!

Also, I did have "some" issues installing libcamera module. I could not get permission with pip to write to the normal directory: (for me) " .local/lib/python3.10/site-packages. So I use this solution so kindly shared by nbaldy: https://github.com/raspberrypi/picamera2/issues/563

So, now the picamera2 and libcamera modules are working. However, it seems that the usb webcam cannot be found by the modules?

Perhaps, there ss there a way to set the correct camera?

Kind regards and any help is very much appreciated :-)

[2024-04-04 10:47:47.322] [ERROR] Whoops! There was an uncaught exception... [2024-04-04 10:47:47.330] [ERROR] PythonShellError: [0:01:35.870026963] [2882]  INFO Camera camera_manager.cpp:284 libcamera v0.2.0+47-46b868fe Traceback (most recent call last): File "/home/bo/MagicMirror/modules/MMM-Face-Reco-DNN/tools/recognition.py", line 40, in picam2 = Picamera2() File "/home/bo/.local/lib/python3.10/site-packages/picamera2/picamera2/picamera2.py", line 242, in init camera_num = self.global_camera_info()[camera_num]['Num'] # 'Num' IndexError: list index out of range Exception ignored in: <function Picamera2.del at 0xffff8bb51f30> Traceback (most recent call last): File "/home/bo/.local/lib/python3.10/site-packages/picamera2/picamera2/picamera2.py", line 404, in del self.close() File "/home/bo/.local/lib/python3.10/site-packages/picamera2/picamera2/picamera2.py", line 604, in close if self._preview: AttributeError: 'Picamera2' object has no attribute '_preview'

at PythonShell.parseError (/home/bo/MagicMirror/modules/MMM-Face-Reco-DNN/node_modules/python-shell/index.js:311:21)
at terminateIfNeeded (/home/bo/MagicMirror/modules/MMM-Face-Reco-DNN/node_modules/python-shell/index.js:193:32)
at ChildProcess.<anonymous> (/home/bo/MagicMirror/modules/MMM-Face-Reco-DNN/node_modules/python-shell/index.js:185:13)
at ChildProcess.emit (node:events:514:28)
at ChildProcess._handle.onexit (node:internal/child_process:294:12) {

executable: 'python3', options: null, script: 'modules/MMM-Face-Reco-DNN/tools/recognition.py', args: [ '--cascade=modules/MMM-Face-Reco-DNN/model/haarcascade_frontalface_default.xml', '--encodings=modules/MMM-Face-Reco-DNN/model/encodings.pickle', '--rotateCamera=-1', '--method=dnn', '--detectionMethod=hog', '--interval=1500', '--output=0', '--outputmm=0', '--extendDataset=True', '--dataset=modules/MMM-Face-Reco-DNN/dataset/', '--tolerance=0.6', '--brightness=0', '--contrast=0', '--resolution=1920,1080', '--processWidth=500' ], exitCode: 1 }

The camera is detected by ubuntu:

bo@mm:~$ ls -ltrh /dev/video* crw-rw----+ 1 root video 81, 2 Apr 4 10:44 /dev/video15 crw-rw----+ 1 root video 81, 1 Apr 4 10:44 /dev/video14 crw-rw----+ 1 root video 81, 6 Apr 4 10:44 /dev/video12 crw-rw----+ 1 root video 81, 0 Apr 4 10:44 /dev/video13 crw-rw----+ 1 root video 81, 3 Apr 4 10:44 /dev/video10 crw-rw----+ 1 root video 81, 10 Apr 4 10:44 /dev/video21 crw-rw----+ 1 root video 81, 8 Apr 4 10:44 /dev/video20 crw-rw----+ 1 root video 81, 12 Apr 4 10:44 /dev/video23 crw-rw----+ 1 root video 81, 7 Apr 4 10:44 /dev/video18 crw-rw----+ 1 root video 81, 13 Apr 4 10:44 /dev/video0 crw-rw----+ 1 root video 81, 9 Apr 4 10:44 /dev/video31 crw-rw----+ 1 root video 81, 5 Apr 4 10:44 /dev/video11 crw-rw----+ 1 root video 81, 4 Apr 4 10:44 /dev/video16 crw-rw----+ 1 root video 81, 11 Apr 4 10:44 /dev/video22 crw-rw----+ 1 root video 81, 14 Apr 4 10:44

bo@mm:~$ lsusb Bus 003 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub Bus 002 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub Bus 001 Device 005: ID 046d:c52b Logitech, Inc. Unifying Receiver Bus 001 Device 009: ID aaec:c021 Multi touch Multi touch overlay device Bus 001 Device 003: ID 1bcf:2284 Sunplus Innovation Technology Inc. Full HD webcam Bus 001 Device 002: ID 2109:3431 VIA Labs, Inc. Hub Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub

The webcam does work in the Ubuntu 22.04

[nischi]

Hi @bojuelkjeldsen

Boah this looks more like an issue in picamera module itself. https://github.com/raspberrypi/picamera2/issues/866 I found this and seems there was something wrong wired, but no more information.

Are you able to test with another camera to check if this is the issue?

[bojuelkjeldsen]

Hi @nischi

Thank you for your reply!

Unfortunately, other web cameras did not work either. I found 3 other cameras to test, one of which was an old Logitech. All worked fine, but not in Picamera/libcamera modules. They gave the same error:

File "/home/bo/.local/lib/python3.10/site-packages/picamera2/picamera2/picamera2.py", line 242, in init camera_num = self.global_camera_info()[camera_num]['Num'] IndexError: list index out of range

I did try with the cameras connected to both USB2 and USB3, with same result. They did work on Ubuntu using the Cheese camera app.

Can USB perhaps be forced only to look for the USB web cam?

Kind regards

Bo

[nischi]

Hi @bojuelkjeldsen

Hmm okay, strange. I need to have a look into the documentation if we can configure some stuff. Need a little bit of time for this.

Kind regards

[nischi]

Hi @bojuelkjeldsen

Proberly i did found a solution for the issue. But not sure if it's working. Can you try something?

In this file /home/bo/MagicMirror/modules/MMM-Face-Reco-DNN/tools/recognition.py

change on line 40 from picam2 = Picamera2() to picam2 = Picamera2(1)

Maybe the usb-camera has another index and 0 (default) is not available. But the error say out of index, so properly it does not work. But worth a try.

[bojuelkjeldsen]

Hi @nischi

Thank you for your response! I did try your suggestions with 1, 2, 3, 4 ... but it did not work.

So, one of my friends (professional IT-guy), also had a look at it, and finally suggested me to start over (again). This time, however, I did use Debian 12 (Bookworm) on my raspberry Pi 4 (Previously, I used Ubuntu 22.04). Also, I did use the virtual environment. AND - after some hours - I also changed the "PythonPath" in the MagicMirror config.js file, as already instructed by you in that file (remember the " " around the path).

SUCCESS!!!

The MMM-Face-Reco-DNN is up and running again!

[nischi]

hi @bojuelkjeldsen

Awesome if had worked 👍