Can not "import camera" by microlite-spiram-cam-esp32-firmware

[yokoyama-flogics]

Hello,

This may be a silly mistake, but please help.

I downloaded microlite-spiram-cam-esp32-firmware (commit: 2aa9d11) and trying to run it on an Espressif ESP-EYE Dev board. (I know it's not tested yet by the project. But the board looks having an OV2640, and the board is supported by ESP-IDF in anyway.)

First of all, booted the MicroPython and I got the following banner.

MicroPython 2aa9d11 on 2022-09-01; ESP32-cam module (microlite) with ESP32

However, when I ran

import camera

I got the error

>>> import camera
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ImportError: no module named 'camera'

And I also checked

>>> help('modules')
__main__          framebuf          uasyncio/lock     upip_utarfile
_boot             gc                uasyncio/stream   uplatform
_onewire          inisetup          ubinascii         urandom
_thread           math              ucollections      ure
_uasyncio         microlite         ucryptolib        uselect
_webrepl          micropython       uctypes           usocket
apa106            neopixel          uerrno            ussl
btree             network           uhashlib          ustruct
builtins          ntptime           uheapq            usys
cmath             onewire           uio               utime
dht               uarray            ujson             utimeq
ds18x20           uasyncio/__init__ ulab              uwebsocket
esp               uasyncio/core     umachine          uzlib
esp32             uasyncio/event    uos               webrepl
flashbdev         uasyncio/funcs    upip              webrepl_setup

It doesn't look it contains camera module.

What am i doing wrong? Any suggestion would be appreciated.

Regards, Atsushi

[uraich]

Hi Atsushi, Just now I managed to compile a new version of tflite-micropthon for the esp32-cam. Unfortunately this was pretty tricky because recent changes of the MicroPython sources break the camera and the tflite module. I managed to flash the binary using idf.py flash but I don't know yet, how to create a combined binary that I can give you access to, for download and flashing. I will give it a try tomorrow. If you dare to try compiling the system yourself, here are some instructions: https://afnog.iotworkshop.africa/do/view/IoT_Course_English/CustomNopMicroPython

[yokoyama-flogics]

Hello @uraich,

Thank you for your reproduction of the issue and investigation!

I understood that the issue came from the original MicroPython source code updates.

And also thanks for the interesting and helpful instructive documentation. I will try compiling the code by myself.

Regards, Atsushi

[uraich]

I updated the instructions with the latest modifications I had to make. When I rebuilt microlite I went step by step: First I commented all includes in micropython-modules/micropython.cmake. This compiles the system with no C-modules. Then I enabled compilation of ulab and checked that I could import it. After that I included the camera driver (first remove the third parameter in MP_REGISTER_MODULE) and I checked that I can import it and I can init the camera. Finally I modified microlite/tensorflow-microlite.c according to jimmo's description and included the module. If you have problems compiling, please let me know.

[yokoyama-flogics]

Hi @uraich,

Thanks again for the detailed explanation.

First, could you let me confirm which version (commit ID) of tensorflow-micropython-examples you refer?

When I checked out the latest master branch of the repo (2c4573d), the submodule micropython looks like below:

bash$ git ls-files -s micropython
160000 517e82eb6b7d93134637b5658132479f4fc260e3 0       micropython

The commit (517e82e) of micropython looks pretty old. In the fact, git log of the commit looks

bash$ git clone git@github.com:micropython/micropython.git
bash$ cd micropython
bash$ git log 517e82e
commit 517e82eb6b7d93134637b5658132479f4fc260e3
Author: Herwin Grobben <h.grobben@aemics.nl>
Date:   Wed Dec 1 17:10:22 2021 +0100

    stm32/fdcan: Fix naming with regards to G4 series.

For example, you told in the Twiki documentation,

The last parameter in MP_REGISTER_MODULE (enable flag) must be removed

However, it doesn't look applying to the above micropython version.

Do we need to delete micropython directory once, and replace it with the latest version of micropython? Could you let me know Git commit ID of micropython which you recently tested?

P.S. One additional question. In the Twiki document, you're writing

• Setup the sub-modules needed for tflm:
  • cd tflite-micro-micropython

Do I read this as cd tensorflow-micropython-examples instead? I couldn't find the directory tflite-micro-micropython anywhere under the top directory.

Thanks and regards, Atsushi

[uraich]

Hi Atsushi, Thanks for going through the instructions and telling me about its deficiencies. If you read the instructions very attentively then you will see that in the first step, where I clone the repository I do it with the command: git clone https://github.com/mocleiri/tensorflow-micropython-examples tflite-micro-micropython This renames tensorflow-micropython-examples to tflite-micro-micropython So... you are right: if you clone it without the last parameter then the folder is called tensorflow-micropython-examples. The micropython version I use is the not the one of the submodule but I pull the very latest version of micropython. In fact the one I have running now is from yesterday! You may delete the micropython directory and clone the new micropython version into it. Checking out master and pulling the newest version may also work. Btw: The same thing is true for ulab. Please make sure it is the very latest version. It has changed last week, due to recent changes of micropython. When I refer to MP_REGISTER_MODULE then this is the call to register a C module within micropython. Please check the last line in micropython-modules/modcamera.c or micropython-modules/tensorflow-microlite.c And yes, this applies only to the newest version of micropython. Hope this helps Uli

[yokoyama-flogics]

Hello @uraich,

Thank you for pointing out my oversight. I didn't see the last argument for the git clone command. Sorry...

I needed some trials, but now I could run the code on Espressif ESP-EYE (ESP32 version). I'm not sure person detection is correctly working, but I will test it tomorrow. (I want an LCD on back.)

As a pitfall for MicroPython beginners (or me), I noticed we needed below prior to idf.py build.

$ cd $(top)/micropython/ports/esp32
$ make BOARD= submodules

As a memorandum, micropython and ulab versions (commit ID) I used are:

micropython: bdbc44474f92db19a40b5f710a140a0bf70fb0ec
micropython-ulab: 5ccfa5cdd9040c2c4219c07b005256427d31ed1c

By the way, when I run import person_detection_cam, I see the following error. (But looks working in anyway.)

E (56309) gpio: gpio_install_isr_service(460): GPIO isr service already installed

And we also sometime see

E (431869) cam_hal: EV-VSYNC-OVF

Finally, the applied diff is as following just FYI.

diff --git a/micropython-modules/microlite/tensorflow-microlite.c b/micropython-modules/microlite/tensorflow-microlite.c
index a9d392e..f7a9c31 100644
--- a/micropython-modules/microlite/tensorflow-microlite.c
+++ b/micropython-modules/microlite/tensorflow-microlite.c
@@ -210,12 +210,13 @@ STATIC const mp_rom_map_elem_t tensor_locals_dict_table[] = {

 STATIC MP_DEFINE_CONST_DICT(tensor_locals_dict, tensor_locals_dict_table);

-const mp_obj_type_t microlite_tensor_type = {
-    { &mp_type_type },
-    .name = MP_QSTR_tensor,
-    .print = tensor_print,
-    .locals_dict = (mp_obj_dict_t*)&tensor_locals_dict,
-};
+MP_DEFINE_CONST_OBJ_TYPE(
+    microlite_tensor_type,
+    MP_QSTR_tensor,
+    MP_TYPE_FLAG_NONE,
+    print, tensor_print,
+    locals_dict, (mp_obj_dict_t*)&tensor_locals_dict
+);

 // audio_frontend

@@ -259,13 +260,14 @@ STATIC const mp_rom_map_elem_t audio_frontend_locals_dict_table[] = {

 STATIC MP_DEFINE_CONST_DICT(audio_frontend_locals_dict, audio_frontend_locals_dict_table);

-const mp_obj_type_t microlite_audio_frontend_type = {
-    { &mp_type_type },
-    .name = MP_QSTR_audio_frontend,
-    .make_new = af_make_new,
-    .print = af_print,
-    .locals_dict = (mp_obj_dict_t*)&audio_frontend_locals_dict,
-};
+MP_DEFINE_CONST_OBJ_TYPE(
+    microlite_audio_frontend_type,
+    MP_QSTR_audio_frontend,
+    MP_TYPE_FLAG_NONE,
+    make_new, af_make_new,
+    print, af_print,
+    locals_dict, (mp_obj_dict_t*)&audio_frontend_locals_dict
+);

 // - microlite interpreter

@@ -409,13 +411,14 @@ STATIC const mp_rom_map_elem_t interpreter_locals_dict_table[] = {

 STATIC MP_DEFINE_CONST_DICT(interpreter_locals_dict, interpreter_locals_dict_table);

-const mp_obj_type_t microlite_interpreter_type = {
-    { &mp_type_type },
-    .name = MP_QSTR_interpreter,
-    .print = interpreter_print,
-    .make_new = interpreter_make_new,
-    .locals_dict = (mp_obj_dict_t*)&interpreter_locals_dict,
-};
+MP_DEFINE_CONST_OBJ_TYPE(
+    microlite_interpreter_type,
+    MP_QSTR_interpreter,
+    MP_TYPE_FLAG_NONE,
+    print, interpreter_print,
+    make_new, interpreter_make_new,
+    locals_dict, (mp_obj_dict_t*)&interpreter_locals_dict
+);

 // main microlite module
@@ -446,5 +449,5 @@ const mp_obj_module_t microlite_cmodule = {
 };

 // Register the module to make it available in Python.
-MP_REGISTER_MODULE(MP_QSTR_microlite, microlite_cmodule, MODULE_MICROLITE_ENABLED);
+MP_REGISTER_MODULE(MP_QSTR_microlite, microlite_cmodule);

diff --git a/micropython-modules/micropython-camera-driver/modcamera.c b/micropython-modules/micropython-camera-driver/modcamera.c
index 07c4622..6f829f9 100644
--- a/micropython-modules/micropython-camera-driver/modcamera.c
+++ b/micropython-modules/micropython-camera-driver/modcamera.c
@@ -956,7 +956,7 @@ const mp_obj_module_t mp_module_camera_system = {
     .globals = (mp_obj_dict_t *)&camera_module_globals,
 };

-MP_REGISTER_MODULE(MP_QSTR_camera, mp_module_camera_system, 1);
+MP_REGISTER_MODULE(MP_QSTR_camera, mp_module_camera_system);

 #endif
diff --git a/examples/person_detection/esp32-cam/person_detection_cam.py b/examples/person_detection/esp32-cam/person_detection_cam.py
index 791a3bc..85bcd21 100644
--- a/examples/person_detection/esp32-cam/person_detection_cam.py
+++ b/examples/person_detection/esp32-cam/person_detection_cam.py
@@ -12,21 +12,26 @@ from machine import Pin,PWM
 try:

        # uncomment for esp32-cam-mb with ov2640 sensor
-       camera.init(0,format=camera.GRAYSCALE,framesize=camera.FRAME_96X96)
+       # camera.init(0,format=camera.GRAYSCALE,framesize=camera.FRAME_96X96)

        # uncomment for the m5 timer camera with ov3660 sensor
        # camera.init(0,format=camera.GRAYSCALE,framesize=camera.FRAME_96X96,
        #            sioc=23,siod=25,xclk=27,vsync=22,href=26,pclk=21,
        #            d0=32,d1=35,d2=34,d3=5,d4=39,d5=18,d6=36,d7=19,
        #            reset=15)
+
+       # uncomment for the ESP-EYE with ov2640 sensor
+    camera.init(0,format=camera.GRAYSCALE,framesize=camera.FRAME_96X96,
+                  sioc=23,siod=18,xclk=4,vsync=5,href=27,pclk=25,
+                  d0=34,d1=13,d2=14,d3=35,d4=39,d5=38,d6=37,d7=36)
 except:
        print("Error when initializing the camera")
        sys.exit()

-# initialize the flash-light LED, it is connected to GPIO 4
-flash_light = PWM(Pin(4))
-# switch it off
-flash_light.duty(0)
+# # initialize the flash-light LED, it is connected to GPIO 4
+# flash_light = PWM(Pin(4))
+# # switch it off
+# flash_light.duty(0)

 # change for m5 timer camera
 # # initialize the flash-light LED, it is connected to GPIO 4
@@ -34,16 +39,24 @@ flash_light.duty(0)
 # switch it off
 # flash_light.off()

+# change for the ESP-EYE with ov2640 sensor
+# initialize the flash-light LED, it is connected to GPIO 4
+# flash_light = Pin(21,Pin.OUT)
+# switch it off
+# flash_light.off()
+
 mode = 1
 test_image = bytearray(9612)

 def handle_output(person):
        if person > 10:
-               flash_light.duty(5)
+               pass
+               # flash_light.duty(5)
                # if m5 timer camera
                # flash_light.on()
        else:
-               flash_light.duty(0)
+               pass
+               # flash_light.duty(0)
                # if m5 timer camera
                # flash_light.off()

In the Python code, LED controls by GPIO haven't been activated yet, but port 21 should be okay for ESP-EYE. I will check it later.

Thanks and Regards, Atsushi

[uraich]

Hi Atsushi, Just for curiosity: Did you manage to get tflite-micropython to run on your esp32-eye? As far as I could see, the camera driver is not included in the esp32s3 version.

[uraich]

Does this board have PSRAM?

[yokoyama-flogics]

Hi @uraich,

Just for curiosity: Did you manage to get tflite-micropython to run on your esp32-eye? As far as I could see, the camera driver is not included in the esp32s3 version.

As you see in the Espressif product page, there are two variants of ESP-EYE. One is equipped with ESP32 and another has ESP32-S3. My boards is the former and I could manage to run the example code by changing camera connection pin assignments only in the Python code, as mentioned in the previous reply.

In addition to the camera evaluation, I also tested no_person_image_data.dat and person_image_data.dat. The result was as same as the result on the example page like:

'not a person' = -25, 'person' = 25
'not a person' = -113, 'person' = 113

I concluded the code was working on the ESP-EYE.

[yokoyama-flogics]

Does this board have PSRAM?

Yes, it does. As mentioned in the Espressif product page, they say it has 8 Mbytes PSRAM.

[uraich]

Thanks for the information. I think we can close the issue now.

[osugiw]

Hi @yokoyama-flogics. How did you solve this issue? I have already tried to change these, but without any results.

Finally, the applied diff is as following just FYI.

diff --git a/micropython-modules/microlite/tensorflow-microlite.c b/micropython-modules/microlite/tensorflow-microlite.c
index a9d392e..f7a9c31 100644
--- a/micropython-modules/microlite/tensorflow-microlite.c
+++ b/micropython-modules/microlite/tensorflow-microlite.c
@@ -210,12 +210,13 @@ STATIC const mp_rom_map_elem_t tensor_locals_dict_table[] = {

 STATIC MP_DEFINE_CONST_DICT(tensor_locals_dict, tensor_locals_dict_table);

-const mp_obj_type_t microlite_tensor_type = {
-    { &mp_type_type },
-    .name = MP_QSTR_tensor,
-    .print = tensor_print,
-    .locals_dict = (mp_obj_dict_t*)&tensor_locals_dict,
-};
+MP_DEFINE_CONST_OBJ_TYPE(
+    microlite_tensor_type,
+    MP_QSTR_tensor,
+    MP_TYPE_FLAG_NONE,
+    print, tensor_print,
+    locals_dict, (mp_obj_dict_t*)&tensor_locals_dict
+);

 // audio_frontend

@@ -259,13 +260,14 @@ STATIC const mp_rom_map_elem_t audio_frontend_locals_dict_table[] = {

 STATIC MP_DEFINE_CONST_DICT(audio_frontend_locals_dict, audio_frontend_locals_dict_table);

-const mp_obj_type_t microlite_audio_frontend_type = {
-    { &mp_type_type },
-    .name = MP_QSTR_audio_frontend,
-    .make_new = af_make_new,
-    .print = af_print,
-    .locals_dict = (mp_obj_dict_t*)&audio_frontend_locals_dict,
-};
+MP_DEFINE_CONST_OBJ_TYPE(
+    microlite_audio_frontend_type,
+    MP_QSTR_audio_frontend,
+    MP_TYPE_FLAG_NONE,
+    make_new, af_make_new,
+    print, af_print,
+    locals_dict, (mp_obj_dict_t*)&audio_frontend_locals_dict
+);

 // - microlite interpreter

@@ -409,13 +411,14 @@ STATIC const mp_rom_map_elem_t interpreter_locals_dict_table[] = {

 STATIC MP_DEFINE_CONST_DICT(interpreter_locals_dict, interpreter_locals_dict_table);

-const mp_obj_type_t microlite_interpreter_type = {
-    { &mp_type_type },
-    .name = MP_QSTR_interpreter,
-    .print = interpreter_print,
-    .make_new = interpreter_make_new,
-    .locals_dict = (mp_obj_dict_t*)&interpreter_locals_dict,
-};
+MP_DEFINE_CONST_OBJ_TYPE(
+    microlite_interpreter_type,
+    MP_QSTR_interpreter,
+    MP_TYPE_FLAG_NONE,
+    print, interpreter_print,
+    make_new, interpreter_make_new,
+    locals_dict, (mp_obj_dict_t*)&interpreter_locals_dict
+);

 // main microlite module
@@ -446,5 +449,5 @@ const mp_obj_module_t microlite_cmodule = {
 };

 // Register the module to make it available in Python.
-MP_REGISTER_MODULE(MP_QSTR_microlite, microlite_cmodule, MODULE_MICROLITE_ENABLED);
+MP_REGISTER_MODULE(MP_QSTR_microlite, microlite_cmodule);

diff --git a/micropython-modules/micropython-camera-driver/modcamera.c b/micropython-modules/micropython-camera-driver/modcamera.c
index 07c4622..6f829f9 100644
--- a/micropython-modules/micropython-camera-driver/modcamera.c
+++ b/micropython-modules/micropython-camera-driver/modcamera.c
@@ -956,7 +956,7 @@ const mp_obj_module_t mp_module_camera_system = {
     .globals = (mp_obj_dict_t *)&camera_module_globals,
 };

-MP_REGISTER_MODULE(MP_QSTR_camera, mp_module_camera_system, 1);
+MP_REGISTER_MODULE(MP_QSTR_camera, mp_module_camera_system);

 #endif

[osugiw]

Hi @yokoyama-flogics. How did you solve this issue? I have already tried to change these, but without any results.

Finally, the applied diff is as following just FYI.

diff --git a/micropython-modules/microlite/tensorflow-microlite.c b/micropython-modules/microlite/tensorflow-microlite.c
index a9d392e..f7a9c31 100644
--- a/micropython-modules/microlite/tensorflow-microlite.c
+++ b/micropython-modules/microlite/tensorflow-microlite.c
@@ -210,12 +210,13 @@ STATIC const mp_rom_map_elem_t tensor_locals_dict_table[] = {

 STATIC MP_DEFINE_CONST_DICT(tensor_locals_dict, tensor_locals_dict_table);

-const mp_obj_type_t microlite_tensor_type = {
-    { &mp_type_type },
-    .name = MP_QSTR_tensor,
-    .print = tensor_print,
-    .locals_dict = (mp_obj_dict_t*)&tensor_locals_dict,
-};
+MP_DEFINE_CONST_OBJ_TYPE(
+    microlite_tensor_type,
+    MP_QSTR_tensor,
+    MP_TYPE_FLAG_NONE,
+    print, tensor_print,
+    locals_dict, (mp_obj_dict_t*)&tensor_locals_dict
+);

 // audio_frontend

@@ -259,13 +260,14 @@ STATIC const mp_rom_map_elem_t audio_frontend_locals_dict_table[] = {

 STATIC MP_DEFINE_CONST_DICT(audio_frontend_locals_dict, audio_frontend_locals_dict_table);

-const mp_obj_type_t microlite_audio_frontend_type = {
-    { &mp_type_type },
-    .name = MP_QSTR_audio_frontend,
-    .make_new = af_make_new,
-    .print = af_print,
-    .locals_dict = (mp_obj_dict_t*)&audio_frontend_locals_dict,
-};
+MP_DEFINE_CONST_OBJ_TYPE(
+    microlite_audio_frontend_type,
+    MP_QSTR_audio_frontend,
+    MP_TYPE_FLAG_NONE,
+    make_new, af_make_new,
+    print, af_print,
+    locals_dict, (mp_obj_dict_t*)&audio_frontend_locals_dict
+);

 // - microlite interpreter

@@ -409,13 +411,14 @@ STATIC const mp_rom_map_elem_t interpreter_locals_dict_table[] = {

 STATIC MP_DEFINE_CONST_DICT(interpreter_locals_dict, interpreter_locals_dict_table);

-const mp_obj_type_t microlite_interpreter_type = {
-    { &mp_type_type },
-    .name = MP_QSTR_interpreter,
-    .print = interpreter_print,
-    .make_new = interpreter_make_new,
-    .locals_dict = (mp_obj_dict_t*)&interpreter_locals_dict,
-};
+MP_DEFINE_CONST_OBJ_TYPE(
+    microlite_interpreter_type,
+    MP_QSTR_interpreter,
+    MP_TYPE_FLAG_NONE,
+    print, interpreter_print,
+    make_new, interpreter_make_new,
+    locals_dict, (mp_obj_dict_t*)&interpreter_locals_dict
+);

 // main microlite module
@@ -446,5 +449,5 @@ const mp_obj_module_t microlite_cmodule = {
 };

 // Register the module to make it available in Python.
-MP_REGISTER_MODULE(MP_QSTR_microlite, microlite_cmodule, MODULE_MICROLITE_ENABLED);
+MP_REGISTER_MODULE(MP_QSTR_microlite, microlite_cmodule);

diff --git a/micropython-modules/micropython-camera-driver/modcamera.c b/micropython-modules/micropython-camera-driver/modcamera.c
index 07c4622..6f829f9 100644
--- a/micropython-modules/micropython-camera-driver/modcamera.c
+++ b/micropython-modules/micropython-camera-driver/modcamera.c
@@ -956,7 +956,7 @@ const mp_obj_module_t mp_module_camera_system = {
     .globals = (mp_obj_dict_t *)&camera_module_globals,
 };

-MP_REGISTER_MODULE(MP_QSTR_camera, mp_module_camera_system, 1);
+MP_REGISTER_MODULE(MP_QSTR_camera, mp_module_camera_system);

 #endif

Nevermind this question. Fortunately, I tried to replace the micropython-camera-driver with the original library which is from https://github.com/lemariva/micropython-camera-driver and the result I could import the camera right now.