"no module named 'usocket'" on Raspberry Pico W

[noinskit]

Hi,

I'm trying to use this package in order to run an HTTP Server (Microdot) concurrently with other tasks on Raspberry Pico W. It's failing for me with:

Traceback (most recent call last):
  File "<stdin>", line 42, in <module>
  File "microdot_asyncio.py", line 320, in run
  File "asyncio/core.py", line 297, in run
  File "asyncio/core.py", line 261, in run_until_complete
  File "asyncio/core.py", line 246, in run_until_complete
  File "microdot_asyncio.py", line 278, in start_server
  File "asyncio/stream.py", line 226, in start_server
ImportError: no module named 'usocket'

Am I doing something totally unsupported yet, like maybe only some subset of CircuitPython devices have usocket?

I know that sockets generally work if I use socketpool for screating sockets, so maybe I'm really looking after https://github.com/adafruit/Adafruit_CircuitPython_asyncio/issues/4 ?

I'd appreciate any pointers!

[dhalbert]

CircuitPython doesn't have usocket, but you can use socketpool. It looks like the microdot server is written for MicroPython, not CircuitPython, but you could modify it.

[noinskit]

I believe that usocket is called from within Adafruit_CircuitPython_asyncio directly.

From what I can see microdot server is only using asyncio and its start_server method, which doesn't sound like depending on MicroPython.

This is my attempt at minimal repro without using Microdot:

import asyncio

async def main():
    async def serve(reader, writer):
        pass

    await asyncio.start_server(serve, "0.0.0.0", 80)

asyncio.run(main())

Note that I'm not really initializing wifi in my example, but since I would not be able to pass the initialized socket pool to asyncio anyway, it wouldn't help.

The result:

Traceback (most recent call last):
  File "<stdin>", line 9, in <module>
  File "asyncio/core.py", line 297, in run
  File "asyncio/core.py", line 261, in run_until_complete
  File "asyncio/core.py", line 246, in run_until_complete
  File "<stdin>", line 7, in main
  File "asyncio/stream.py", line 226, in start_server
ImportError: no module named 'usocket'

I'll admit that in the meantime I also tried to make a socketpool.SocketPool object pretend to be the usocket module for the sake of asyncio, but it's currently failing for me onsocket objects from there not having a setsockopt method.

[dhalbert]

You're right, we should change this line: https://github.com/adafruit/Adafruit_CircuitPython_asyncio/blob/a044da1e7f0c6900cc50f86ad8a372c1144e7709/asyncio/stream.py#L152

[noinskit]

Also true, but in my case I think it's about the server part: https://github.com/adafruit/Adafruit_CircuitPython_asyncio/blob/a044da1e7f0c6900cc50f86ad8a372c1144e7709/asyncio/stream.py#L226

[chrismurf]

This bit me tonight with the AdaFruit ESP32 Feather V2 as well, so for anybody new and googling - it's not specific to the Raspberry Pico W at all.

I believe this will also affect open_connection calls.

[jordonsc]

The same applies to the uerrno import on the line above.

[byzantic]

I'm having the same issue here, except that I am trying to use the Wiznet W5500-EVB-Pico.

Now, interestingly, this does have a socket implementation already (adafruit_wiznet5k.adafruit_wiznet5k_socket) .. but how would I set up asyncio to use that instead?

[mcarlson]

This bit me also, which is too bad because the official circuitpython story for not supporing interrupts/multithreading (which I get!) is to use asyncio for applications where you need to do more than one thing at a time. For me, that means going back to micropython as I'm building 'headless' apps that rely on microdot/being able to run an async server.

Not super stoked because the story for not having interrupts in circuitpython is is 'use asyncio' per https://learn.adafruit.com/cooperative-multitasking-in-circuitpython-with-asyncio/overview

But, the circuitpython asyncio doesn't work with microdot because they don't implement start_server. So I'm blocked... Please fix this!

[sjev]

.... this issue is still relevant and affects probably all ports. I was making a switch from Micropython to Circuitpython, but tripped on:

asyncio.create_task(asyncio.start_server(serve_client, "0.0.0.0", 80))

crashes with

Traceback (most recent call last):
  File "asyncio/core.py", line 261, in run_until_complete
  File "asyncio/stream.py", line 225, in start_server
ImportError: no module named 'usocket'

Major bummer :disappointed: , given that according to docs start_server should be supported. Shouldn't this be in test coverage?

[mcarlson]

I believe it really should be fixed. For me, I wasn’t able to continue with micropython because of it. Which is a double bummer because I’ve had to backport so so many drivers. I hope this can be fixed soon, it will make embedded servers much more viable on micropython.

Thank you!

Regards, Max Carlson

On May 8, 2024, at 11:10 PM, Jev Kuznetsov @.***> wrote:

.... this issue is still relevant and affects probably all ports. I was making a switch from Micropython to Circuitpython, but during testing running: asyncio.create_task(asyncio.start_server(serve_client, "0.0.0.0", 80)) crashes with Traceback (most recent call last): File "asyncio/core.py", line 261, in run_until_complete File "asyncio/stream.py", line 225, in start_server ImportError: no module named 'usocket'

Major bummer 😞 , given that according to docs start_server should be supported. Shouldn't this be in test coverage? — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you commented.Message ID: @.***>

[mattytrentini]

I believe it really should be fixed. For me, I wasn’t able to continue with micropython because of it.

Just to clarify @mcarlson, I think you mean CircuitPython here? I've been using microdot on MicroPython for years with no issue whatsoever...

[dhalbert]

Have any of you tried the source version and just fixed the import names? I started a PR to do that. There are only a few lines to change. It might still be the case that something doesn't work.

[mcarlson]

Yes, sorry I misspoke - I meant CircuitPython. I’ve had to stick with Micropython so I have asynchronous http support and my embedded server leaves time for other tasks to run. It’s quite amazing what the pico can do!

Regards, Max Carlson

On May 9, 2024, at 8:16 AM, Matt Trentini @.***> wrote:

I believe it really should be fixed. For me, I wasn’t able to continue with micropython because of it. Just to clarify @mcarlson, I think you mean CircuitPython here? I've been using microdot on MicroPython for years with no issue whatsoever... — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you were mentioned.Message ID: @.***>

[sjev]

@dhalbert

Have any of you tried the source version and just fixed the import names? I started a PR to do that. There are only a few lines to change. It might still be the case that something doesn't work.

You mean rebuilding CP? Could you please link to your PR?

[mcarlson]

Please send a link to the PR/branch/commit and I’d be happy to give this a test! I’d so love to be using CircuitPython… Thank you!

Regards, Max Carlson

On May 9, 2024, at 5:47 PM, Jev Kuznetsov @.***> wrote:

Have any of you tried the source version and just fixed the import names? I started a PR to do that. There are only a few lines to change. It might still be the case that something doesn't work. You mean rebuilding CP? Could you please link to your PR? — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you were mentioned.Message ID: @.***>

[dhalbert]

You mean rebuilding CP? Could you please link to your PR?

I just mean changing the names in this library. Just get the Python (not .mpy) version of this library and edit it so the import names match. e.g. change import usocket as socket to import socketpool as socket, etc.

[sjev]

@dhalbert - ... couple of hours further it does not seem that simple.

I replaced import usocket as socket in start_server by this:

import wifi
import socketpool

socket = socketpool.SocketPool(wifi.radio)

It helpled a bit - the server could be started. only to crash later with

Traceback (most recent call last):
  File "/lib/asyncio/core.py", line 261, in run_until_complete
  File "webserver.py", line 16, in serve_client
  File "/lib/asyncio/stream.py", line 100, in readline
AttributeError: 'Socket' object has no attribute 'readline'

BTW, there is more smelly stuff in stream.py , like

    from uerrno import EINPROGRESS  # there is no `uerrno` in circuitpython
    import socketpool as socket  # these are not equivalent. 

It does not not feel right to keep hacking further without proper test cases in place. That would not be trivial to do, because we'd need something like descent mocks first on a unix port.

[mcarlson]

Would https://github.com/micropython/micropython/blob/master/tests/run-tests.py be useful as the basis for a testing framework?

Regards, Max Carlson

On May 10, 2024, at 6:39 AM, Jev Kuznetsov @.***> wrote:

@dhalbert - ... couple of hours further it does not seem that simple. I replaced import usocket as socket in start_server by this: import wifi import socketpool

socket = socketpool.SocketPool(wifi.radio)

It helpled a bit - the server could be started. only to crash later with Traceback (most recent call last): File "/lib/asyncio/core.py", line 261, in run_until_complete File "webserver.py", line 16, in serve_client File "/lib/asyncio/stream.py", line 100, in readline AttributeError: 'Socket' object has no attribute 'readline'

BTW, there is more smelly stuff in stream.py , like from uerrno import EINPROGRESS # there is no uerrno in circuitpython import socketpool as socket # these are not equivalent.

It does not not feel right to keep hacking further without proper test cases in place. That would not be trivial to do, because we'd need descent mocks first on a unix port. — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you were mentioned.Message ID: @.***>

[sveinungkb]

Any path forward here?

[dhalbert]

I think there are two things going on here:

1. The u prefix needs to be removed in all places where it's used.
2. microdot uses .readline() on a Socket, which is not standard in CPython and is not implemented in CircuitPython.

If your goal is to just have an HTTP server, then there is https://github.com/adafruit/Adafruit_CircuitPython_HTTPServer, but it is not async.

[dhalbert]

We are just starting to merge MicroPython v1.21 through v1.23 into CircuitPython. A side effect of this will be updating this asyncio library from the version that is in https://github.com/micropython/micropython/tree/master/extmod/asyncio.

I talked with @tannewt briefly about this in discord. We need start_server() if we don't have it already. See https://github.com/miguelgrinberg/microdot/issues/214 as well.

[mcarlson]

I would love it if there was because circuitpython is so appealing in so many ways!

Personally, I’ve had to stick with micropython because of these issues. I need an embedded web server!

Regards, Max Carlson

On Jul 12, 2024, at 2:19 AM, Sveinung Kval Bakken @.***> wrote:

Any path forward here? — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you were mentioned.Message ID: @.***>

[dhalbert]

I need an embedded web server!

Have you looked at https://github.com/adafruit/Adafruit_CircuitPython_HTTPServer, or do you need the async support?

[mcarlson]

Yes, I need the async support. Otherwise I can’t easily use sensors, control lighting etc. etc.

Regards, Max Carlson

On Jul 12, 2024, at 11:09 AM, Dan Halbert @.***> wrote:

I need an embedded web server! Have you looked at https://github.com/adafruit/Adafruit_CircuitPython_HTTPServer, or do you need the async support? — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you were mentioned.Message ID: @.***>

[sveinungkb]

I'm doing several tasks, for sensors etc, but anything more than a second or so between each check if the socket the web experience seems very bad.

There's async support in the socket itself in MP, but the same doesn't seem present in CP. See the socket callback registered here.

https://github.com/orgs/micropython/discussions/12219#discussioncomment-6713393

The code to register, check and invoke these subscriptions are not present in CP's socket.c as far as I can tell.

I'm getting an exception when trying to run the same code as this only expects integer or buffer as 3rd arguments. Socket.c:346 If we could register a callback, we could queue the task to process the request was my angle.

[mattytrentini]

I would love it if there was because circuitpython is so appealing in so many ways!

CircuitPython is great - and I'm not trying to convince you otherwise! - but I am interested in why you'd prefer CircuitPython over MicroPython here? Perhaps you've had a bad experience with MicroPython? Which may give us MicroPython folks a chance to learn...

Not to take anything away from CircuitPython, but I'm always interested in improving the experience for MicroPython users.

[mcarlson]

No bad experience with Micropython, It’s an amazing piece of software! Miles and miles better than the old C++ AVR/arduino days the last time I tuned into this channel…

I think they’re both great for different things. Circuitpython has such a wealth of drivers for different things, e.g. rgb(w) light strip support is far more advanced than what’s available for Micropython, e.g. I had to spend a lot of time and effort improvinghttps://github.com/blaz-r/pi_pico_neopixel/commits/main/ to get it closer to the way I needed it to work - including backporting a bunch of work already done in circuitpython to have better RPI PIO drivers, improve APIs etc. etc.

If I was in the circuitpython universe I would have been there already. The same seems to be true with drivers in general.

I insist on having a good async web server for headless operation and stable bluetooth support for device discovery and federation, so I’ve stuck with micropython. The sticking point for me really has been inadequate support for https://github.com/miguelgrinberg/microdot ( which has been really, really great.) in circuitpython...

Micropython seems to be more robust and stable for asynchronous ops and ‘low-level’ work, while Circuitpython has much better support for easily and quickly supporting various devices and sensors.

I really wish there’d never been a fork - that was a huge mistake. The Micropython people should have listened to the wisdom of Lady Ada, and now they’re having to do a lot of the same kinds of hardware abstractions she wanted anyway…

Regards, Max Carlson

On Jul 12, 2024, at 3:36 PM, Matt Trentini @.***> wrote:

I would love it if there was because circuitpython is so appealing in so many ways! CircuitPython is great - and I'm not trying to convince you otherwise! - but I am interested in why you'd prefer CircuitPython over MicroPython here? Perhaps you've had a bad experience with MicroPython? Which may give us MicroPython folks a chance to learn... Not to take anything away from CircuitPython, but I'm always interested in improving the experience for MicroPython users. — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you were mentioned.Message ID: @.***>

[mcarlson]

That’s been my experience also. It’s very frustrating discovering this several days into a project when everything is working great and you try to go headless...

On Jul 12, 2024, at 3:24 PM, Sveinung Kval Bakken @.***> wrote:

I'm doing several tasks, for sensors etc, but anything more than a second or so between each check if the socket the web experience seems very bad. There's async support in the socket itself in MP, but the same doesn't seem present in CP. See the socket callback registered here. https://github.com/orgs/micropython/discussions/12219#discussioncomment-6713393 The code to register, check and invoke these subscriptions are not present in CP's socket.c as far as I can tell. I'm getting an exception when trying to run the same code as this only expects integer or buffer as 3rd arguments. Socket.c:346 If we could register a callback, we could queue the task to process the request was my angle. — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you were mentioned.Message ID: @.***>

[tannewt]

I would love it if there was because circuitpython is so appealing in so many ways!

CircuitPython is great - and I'm not trying to convince you otherwise! - but I am interested in why you'd prefer CircuitPython over MicroPython here? Perhaps you've had a bad experience with MicroPython? Which may give us MicroPython folks a chance to learn...

Not to take anything away from CircuitPython, but I'm always interested in improving the experience for MicroPython users.

Please don't ask this in an issue. You can follow up outside of the issue instead if you want. Here it is off-topic.