shariltumin / esp32-cam-micropython-2022

MicroPython esp32-cam firmware with camera support compiled with esp-idf-4.4.0. Python script files for live streaming
MIT License
174 stars 33 forks source link

No module named 'espnow'. #25

Open aniket-2210 opened 1 year ago

aniket-2210 commented 1 year ago

I am using esp32 cam with micropython for my drone and I want to use esp-now protocol for data transfer between multiple esp32 camera and Raspberry pi. But I cannot import espnow package using this firmware. I had researched on the internet and they say espnow is an inbuilt package that comes with micropython firmware of esp32cam. Since your firmware is customized, I request you to upload espnow support with it. I hope there is a solution to this.

shariltumin commented 1 year ago

The payload in ESP-NOW is limited to 250 bytes. It is designed to send small messages rather than streaming images. Aside from that, ESP-NOW only works with ESP MCUs. For example, you cannot use ESP-NOW with a Raspberry Pi. Certain functionalities are not included in the firmware due to RAM limitations on the esp32-cam. Based on your goals, I believe AP mode WiFI on the esp32-cam with UDP broadcast could be a viable solution for your drone project.

I do not intend to add espnow to the firmware in the near future unless I have a compelling reason to do so.

Have you tried sending, say, 25KByte between esp32 boards using ESP-NOW? You can try this with vanilla MicroPython-V1.20. It would be interesting to know if you can send a jpeg file and at what rate.

aniket-2210 commented 1 year ago

Thank you Sir for your reply, so here I want the live video footage from two to three esp32cam simultaneously, and all those data I want to send on Raspberry Pi to do image processing for further application. Now as you know esp32cam transmits data over wifi and is not suitable for a wired connection to transfer data. My plan is to make Raspberry Pi an Access point and connect all 3 esp32cam wifi to the Raspberry Pi Hotspot network. If this is possible then can I transfer live footage data from multiple esp32cam to one on-board computer simultaneously? I hope you understand my plan and help me to reach my goal, please suggest me a way to do this.

Note: I have to do it without the internet.

And I apologize for my bad English.

shariltumin commented 1 year ago
  1. Let RPI be the access point, and the three ESP1, ESP2, ESP3 are stations to the AP.
  2. Run 3 TCP servers on RPI listening on ports 8001, 8002 and 8003.
  3. Connect each ESP to RPI.
  4. Write a TCP client on each ESP.
  5. Once connected to the WiFi, ESP1 will connect to RPI at TCP:8001, ESP2 at 8002, and ESP3 at 8003.
  6. Each ESP can then send data to RPI.

What you need to know.

  1. How a station joins a WiFi access point - network module
  2. TCP/IP socket programming - socket module
  3. Thread programming if you want to run all 3 servers in one program

I hope this helps. I wish you good luck!

aniket-2210 commented 1 year ago

ok sir, so I am working on it and I guess I am very close to my goal. Can you please tell me if just like opencv has imencode() function, what function does the camera have in micropython firmware? I am asking this because I have to encode data so that I can transfer it through the hotspot. I hope this is the right method I am working on. If there is an alternative method which is reliable and easy please suggest me.

I used some codes from the internet and I would like to share the client code that will run on the esp32 cam.

Server code for Raspberry pi

import socket
import os
import threading
import numpy
import pickle
import cv2

s1 = socket.socket(socket.AF_INET,socket.SOCK_DGRAM)
port1 = 8001

s1.bind(('192.168.137.1',port1))

while True:
    x=s1.recvfrom(1000000)
    clientip = x[1][0]
    data=x[0]
    print(data)
    data=pickle.loads(data)
    print(type(data))
    data = cv2.imdecode(data, cv2.IMREAD_COLOR)
    cv2.imshow('server', data) #to open image
    if cv2.waitKey(10) == 13:
        break
cv2.destroyAllWindows()

I am using opencv in Raspberry Pi for doing image processing.

Client code for esp32

import camera
import socket
import pickle
import os  
s1=socket.socket(socket.AF_INET,socket.SOCK_DGRAM)
#s1.setsockopt(socket.SOL_SOCKET, socket.SO_SNDBUF, 10000000)
port1 = 8001
s1.bind(('192.168.137.1',port1))

camera.init()
while True:    
    photo = camera.capture()      
    buffer = camera.imencode(".jpg", photo,[int(cv2.IMWRITE_JPEG_QUALITY),30])    
    x_as_bytes = pickle.dumps(buffer)       
    s1.sendto(x_as_bytes,("192.168.137.1", port1))    
    if camera.waitKey(10) == 13:              
          break  
camera.destroyAllWindows()
cap.release()

I belive that this code have many errors

shariltumin commented 1 year ago

No, the camera module is nothing like OpenCV. OpenCV runs on a server with sufficient resources.

These are the methods in the camera module:

>>> import camera
>>> dir(camera)
['__class__', '__name__', '__dict__', 'aecvalue', 'aelevels', 'agcgain', 'brightness', 'capture', 'capture_bmp', 'capture_jpg', 'conf', 'contrast', 'deinit', 'flip', 'framesize', 'init', 'mirror', 'pixformat', 'quality', 'saturation', 'speffect', 'whitebalance']

The camera module is merely an interface to the OV2640.

Sorry if I sound harsh, but copying and pasting code examples from the Internet is not the way to go.

I doubt your code will even work. My advice is to start slowly.

  1. Make an RPI UDP server and an ESP UDP client communicate with each other.
  2. Once you've mastered 1, try sending image data from ESP camera to RPI.
  3. After step 2, try to save raw image data from ESP to an RPI file.
  4. Once you have the jpg file on RPI, you can do whatever you want with it usingcv2, example cv2.imshow().

Please keep in mind that GitHub issues are not intended to be used to get programming help.

Hints:

  1. The ESP must connect to the Wi-Fi network provided by the RPI access point.
  2. Only the UDP server requires s.bind(addr).
  3. There is no need to 'pickle' data before sending it to the server.
  4. The camera module does not support opencv.
  5. OpenCV cv2 is extensive and complicated.
shariltumin commented 1 year ago

Please have a look at the video_streaming example. Hopefully this will help you with your project.

biemster commented 9 months ago

To add some extra info to this thread since I'm trying to do the same, espnow was added only semi-recently to the dev branch of MicroPython (https://github.com/micropython/micropython/pull/6515) and just did not make it into the latest (1.20 as of this writing) release. ESP-NOW on linux works great in userspace: https://github.com/thomasfla/Linux-ESPNOW, and should be able to provide significantly lower latency compared to a WiFi network (but I still have to test this). And easier to set up of course, for video streaming it could just firehose everything out.

So if this could be added to this awesome firmware, would be much appreciated.