jczic / MicroWebSrv2

The last Micro Web Server for IoTs (MicroPython) or large servers (CPython), that supports WebSockets, routes, template engine and with really optimized architecture (mem allocations, async I/Os). Ready for ESP32, STM32 on Pyboard, Pycom's chipsets (WiPy, LoPy, ...). Robust, efficient and documented!
https://github.com/jczic/MicroWebSrv2
MIT License
659 stars 97 forks source link

cannot load module websockets #21

Open kthanigaivel opened 4 years ago

kthanigaivel commented 4 years ago

Traceback (most recent call last): File "", line 148, in File "MicroWebSrv2/microWebSrv2.py", line 136, in LoadModule MicroWebSrv2Exception: Cannot load module "WebSockets".

fberier commented 4 years ago

I got the same with esp32 module

jczic commented 4 years ago

Hello, Is this exception raised from main.py demo? (It's important to use from MicroWebSrv2 import * to import the lib and modules).

bennyzen commented 4 years ago

yes, it's raised from main.py using freshly cloned demo. any clue?

jczic commented 4 years ago

Ok, could you try this code in terminal and give me the result please:

from MicroWebSrv2  import *
print(MicroWebSrv2.__module__)

And also, could you check if __import__ function exists in your MicroPython version.

Thank you for your help!

ianmxyz commented 4 years ago

I'm having the same problem, this is the result of doing as you ask:

`>>>

from MicroWebSrv2 import * print(MicroWebSrv2.module) MicroWebSrv2.microWebSrv2 `

ianmxyz commented 4 years ago

And also, could you check if __import__ function exists in your MicroPython version.

I guess not . . .

(for some reason some of the characters are stripped out once I post this comment, including underscore and anything within quotes. Posting it as code doesn't help. Capture )

from import import * Traceback (most recent call last): File "", line 1, in ImportError: no module named 'import

jczic commented 4 years ago

You must not import __import__ but just check if this native function exists just by typing __import__ in your terminal.

ianmxyz commented 4 years ago

It appears that it does.

Capture

CwbhX commented 4 years ago

aaaaand I have the same issue :) and the WebSockets part is the coolest part to me haha. Also an ESP32

CwbhX commented 4 years ago

Well I got some updates: memory allocation failed, allocating 88 bytes I put a simple: except Exception as e: print(e) before your raise MicroWebSrv2Exception('Cannot load module "%s".' % modName)

Any clue how we might be able to fix this, doesn't seem like too many bytes to allocate for an ESP32

Update: I put some debug mem_free() calls in the LoadModule func to see how much ram in the heap is available... and in the call before WebSockets ( the PyhtmlTemplate load module) and the websockets load call I have this now:

MPY: soft reboot

       ---------------------------
       - Python pkg MicroWebSrv2 -
       -      version 2.0.6      -
       -     by JC`zic & HC2     -
       ---------------------------

 + [@WebRoute] GET /test-redir
 + [@WebRoute] GET /test-post (TestPost1/2)
 + [@WebRoute] POST /test-post (TestPost2/2)

Available memory (bytes): 26416
Available memory (bytes): 31920
memory allocation failed, allocating 299 bytes
Traceback (most recent call last):
  File "main.py", line 152, in <module>
  File "MicroWebSrv2/microWebSrv2.py", line 139, in LoadModule
MicroWebSrv2Exception: Cannot load module "WebSockets".
MicroPython v1.12-35-g10709846f on 2020-01-06; ESP32 module with ESP32

Update AGAIN woo (the woo doesn't mean its fixed): So this time I added some of those classic print lines to see where we encounter our lovely memory allocation failed problem and I'm here to report it is the line: module = getattr(__import__(modPath).mods, modName) ... and then it gets a little too involved in WebSockets for me at 1AM so, Bonjour cause I guess its 7AM there. Thanks @jczic for the awesome library nonetheless :)

ianmxyz commented 4 years ago

I had to comment out the call to start websockets before it would work AND change self.slotCount to 8.

def SetEmbeddedConfig(self) : self._validateChangeConf() self._backlog = 8 self._slotsCount = 8 self._slotsSize = 1024 self._keepAlloc = True self._maxContentLen = 16*1024

ianmxyz commented 4 years ago

Also call to PyhtlmlTemplate is commented out, so I guess the whole LoadModule function isn't working.

CwbhX commented 4 years ago

@ianmxyz I'm sorry, where is this

ianmxyz commented 4 years ago

@ianmxyz I'm sorry, where is this

The SetEmbeddedConfig is in microWebSrv2.py around line 340.

CwbhX commented 4 years ago

Sooo, I got it working on my 4Mb PSRAM ESP32 and the appropriate version of Micropython. Therefore, I really think it is the library just using too many resources :/

ianmxyz commented 4 years ago

Sooo, I got it working on my 4Mb PSRAM ESP32 and the appropriate version of Micropython. Therefore, I really think it is the library just using too many resources :/

How did you get it working exactly? Using Micropython2? Also did you get websockets working?

CwbhX commented 4 years ago

@ianmxyz I'm using an ESP32 with an extra (4Mb) PSRAM module soldered onto the board. The version of micropython is the latest on the website: http://micropython.org/download#esp32 where it says "GENERIC-SPIRAM". But the standard ESP32s I have seemingly don't have enough memory for this library in its current form. And yes everything worked except I didn't have any of the large files on there. Tbh, I'd be curious to try this on my K210 board some time

ElHyperion commented 4 years ago

I'm having no problems with running MWS2 on my ESP32 with the default 2Mb memory space. Did you "freeze" the module by building the MPY firmware yourself instead of using the prebuilt binaries from the website? That's the only way I could run it, I also got mem allocation errors before when I added the library into the filesystem and imported it during runtime.

CwbhX commented 4 years ago

@ElHyperion No, I looked into making them into mpy files but I haven't done that before and it seemed like it doesn't always help tooo much, and I wasn't sure if it was a bug at first or actually memory allocation errors, but good to know that made it work for you! I'm hoping to use this library with my BMS haha

ianmxyz commented 4 years ago

@ianmxyz I'm using an ESP32 with an extra (4Mb) PSRAM module soldered onto the board. The version of micropython is the latest on the website: http://micropython.org/download#esp32 where it says "GENERIC-SPIRAM". But the standard ESP32s I have seemingly don't have enough memory for this library in its current form. And yes everything worked except I didn't have any of the large files on there. Tbh, I'd be curious to try this on my K210 board some time

I've given up and reverted to developing my project on an Onion Omega 2+. Perhaps an overkill but it already runs a webserver so I can keep it all in one place. But it only has 2 channels of hardware PWM and I need 4, I'm using the PWM expansion at the moment. However as ultimately I'll need a 4 channel LED driver I can handle that with I2S in any case. I'm sure the ESP32 is perfectly capable of what I need it to do ultimately.

joseshiru commented 4 years ago

Got the same problem with a ESP-32 + 4MB.

jczic commented 4 years ago

Hm.. have you tested by compiling the firmware with the lib (MicroWebSrv2) included in /frozen?

bennyzen commented 4 years ago

yep, I've tried it with most recent firmware and it just works. confirmed

bennyzen commented 4 years ago

for those of you struggling with the process: just put the whole MicroWebSrv2 folder into ports/esp32/modules, hit make submodules and then make deploy to recompile MicroPython and reflash. Now you should have the webserver baked into your current firmware.

SolarDaniel commented 4 years ago

Concerning LoadModule and other issues

Well, I'm new to Python and github, but maybe my experiences help some other newbies. I'm running microWebSrv2 on a Pycom WiPy3 with 4MB flash. It did not work out of the box, needed some debugging and changes. Pycom microPython's implementation does not support:

Change directory 'libs' to 'lib', Pycom's default.

Changes in file microWebSrv2.py

#from .             import *
#from .httpRequest  import HttpRequest
from urlUtils import *
from XAsyncSockets import *
from httpRequest  import HttpRequest

and in file httpRequest.py

#from   .              import *
#from   .httpResponse  import HttpResponse
from urlUtils import *
from webRoute import *
from httpResponse  import HttpResponse

LoadModule in file microWebSrv2.py didn't work. I don't know the reason, but following change helped:

 #module   = getattr(__import__(modPath).mods, modName)
 module   = getattr(__import__(modPath), modName)

When using route arguments, request will fail, due to regex's missing support for above mentioned features. Following change helped:

#regex += '/([\\w.]*)'
regex += '/([A-Za-z0-9_.]*)'

Finally the webserver runs stable and fast and there is plenty of code space for other tasks.

@jczic Thank you Jean-Christophe for this awesome piece of code.

berkut0 commented 3 years ago

Hi, any solution for ESP32? Websocket still not working due failing of module import

instruction in https://github.com/jczic/MicroWebSrv2/issues/21#issuecomment-623881378 doesn't work for my ESP32 Devkitv1

https://github.com/jczic/MicroWebSrv2/issues/21#issuecomment-579795744 sounds great, but too complicated at this point of research (and for windows machines)

CwbhX commented 3 years ago

@berkut0 do you have an ESP32 with additional PSRAM? I found that required

berkut0 commented 3 years ago

@berkut0 do you have an ESP32 with additional PSRAM? I found that required

@CwbhX Looks like it has only flash memory and not PSRAM. Is it the reason that cause the failure?

oniram88 commented 3 years ago

For everyone that know a little of Docker, this is my simple file to build an image that contains the firmware with the server builded inside:

FROM python:3.9.5-buster

RUN apt-get update && apt-get install -yq \
    build-essential \
    libffi-dev \
    git \
    pkg-config \
    python3-libusb1 \
    git wget libncurses-dev flex bison gperf cmake ninja-build ccache libffi-dev libssl-dev

RUN git clone -b v4.0.2 --recursive https://github.com/espressif/esp-idf.git
RUN /esp-idf/install.sh

RUN git clone -b v1.15 https://github.com/micropython/micropython.git
RUN cd /micropython && make -C mpy-cross

# esp32
RUN git clone -b v2.0.6 --recursive https://github.com/jczic/MicroWebSrv2.git
RUN cp -r /MicroWebSrv2/MicroWebSrv2 /micropython/ports/esp32/modules/MicroWebSrv2
RUN cd /micropython/ports/esp32 && /bin/bash -c 'source /esp-idf/export.sh; make submodules; make'

and with the docker compose file you can copy outside and build in one command

version: "3.8"

services: 
  builder:
    build: .
    volumes: 
      - "./build:/builded"
    command: "cp /micropython/ports/esp32/build-GENERIC/firmware.bin /builded/."
wattzor commented 2 years ago

Any solution if you do not have the extra ram? i go the ESP32 TTGO display and its failing on the import of WebSockets

FriedrichUthe commented 2 years ago

There was nothing I could do about it, endet Up using a different module. Cannot remember which one out of my head but would look it up over the weekend if it helps.

Holen Sie sich Outlook für Androidhttps://aka.ms/AAb9ysg


From: wattzor @.> Sent: Thursday, October 7, 2021 6:47:45 PM To: jczic/MicroWebSrv2 @.> Cc: FriedrichUthe @.>; Comment @.> Subject: Re: [jczic/MicroWebSrv2] cannot load module websockets (#21)

Any solution if you do not have the extra ram? i go the ESP32 TTGO display and its failing on the import of WebSockets

— You are receiving this because you commented. Reply to this email directly, view it on GitHubhttps://github.com/jczic/MicroWebSrv2/issues/21#issuecomment-937974055, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AMQGAQIBNUEYX5FKWBG265TUFXFLDANCNFSM4KAKN4MQ. Triage notifications on the go with GitHub Mobile for iOShttps://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Androidhttps://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.