Add uasyncio version of urequests (with patch)

[GM-Script-Writer-62850]

Patch: urequests.zip Attachment includes both before/after scripts of the current version of urequests as well as a diff.patch file This also includes a copy of this with the patch applied for issue #546

Using this test code shows my uasyncio urequests patch works

import uasyncio_urequests as urequests
import uasyncio

# [insert code to connect to network here]

async def test():
    while True:
        print("testing")
        await uasyncio.sleep(1)
uasyncio.create_task(test())
async def main():
    while True:
        r=await urequests.get("http://10.0.0.75/temp.json")
        print(r.status_code,r.content)
        #await uasyncio.sleep(3)
uasyncio.run(main())

[jimmo]

Thanks @GM-Script-Writer-62850 -- this is neat and definitely supporting a more full-featured asyncio http client is something we need for MicroPython.

I'm not sure adding an async version of urequests makes sense though if the library it's based on (requests -- https://requests.readthedocs.io/en/latest/) doesn't itself support asyncio.

I think the effort would be better spent turning this into a micro implementation of aiohttp (or any of the other asyncio-based Python http libraries).

[GM-Script-Writer-62850]

i did look at this one, but it did not look to support sending post data (massive deal breaker for my use case) https://github.com/micropython/micropython-lib/blob/master/micropython/uaiohttpclient/uaiohttpclient.py in my current project i only make a single GET request at boot, i use POST for sending data to log and backing up config data

[mattytrentini]

request takes an HTTP method as the first parameter so something like this should work:

resp = yield from aiohttp.request("POST", url)

Mind you, just throwing it out there, I'd love to have a MicroPython port of httpx if you feel like going down that path... 😄

[GM-Script-Writer-62850]

aiohttp.request("POST", url) now how do i send data? i have a URL and a method, but no way to send post data and what about setting the content type header (eg: application/x-www-form-urlencoded or applicatoion/json)?

[mattytrentini]

Errr, umm, yes, that seems distinctly unimplemented. 🙄

[GM-Script-Writer-62850]

i realized today i could also just make a non-blocking post request that will be delayed till my sleep call happens

async def post(json_str,comment=""):
    await sleep(0)
    r=urequests.post(config.remote_url,data=json_str,headers={"Content-type":"application/json"})
    if r.status_code != 200:
        print("------ERROR------")
        # probably should log a few errors in memory so i can check if there are errors occurring later
    print(comment,r.status_code,"-", r.content)
    r.close()

uasyncio.create_task(post(data,"Update backup config data:"))

Could combine this with my modded uasyncio_urequests lib, just not sure if it is even a good idea to make it possible to have multiple open requests while at the same time having a web server process running as the same time (i think the pico w has a limit of 4 open sockets)

[jimmo]

This is not a non-blocking post request. It's just a deferred post request that is still blocking. Your event loop will still pause while one HTTP request happens and there will only be one concurrent request.

[GM-Script-Writer-62850]

i know it is just deferred, i am just not sure it would be a good idea to do it truly non-blocking, in my use case this would max at 2 post request running at the same time

the way my code is setup i have a entire 2 seconds to make post request w/out interfering with any other code execution at all (baring once a day events and a client actively connected) and this window will be followed by a 750ms window

things to consider:

• would trying to open a second socket to the same server cause a error? (2 deferred post running yielding the socket to each other)
  • it would be better to make a post data array and then post all in the same connection, with it being deferred that would not be that hard to do (assuming i keep it blocking; could result in race conditions otherwise)
• What is the likely hood i could exceed the max socket limit of my wifi chip

[bulletmark]

@GM-Script-Writer-62850 FYI, I am looking for an asyncio http client so found this and tried it out but it does not actually do the requests in parallel. It works the same as ordinary requests.

[GM-Script-Writer-62850]

Tired this one? https://github.com/Carglglz/asyncmd/blob/main/async_modules/async_urequests/async_urequests.py

[Carglglz]

@bulletmark @GM-Script-Writer-62850 you may want to try this #752, just remove ssl=... parameter in

..
reader, writer = await asyncio.open_connection(host, port, ssl=ssl)
...

and it should work 👍🏼, e.g.

import uaiohttpclient as aiohttp
import asyncio

 async def fetch(client):
     async with client.get("http://micropython.org") as resp:
         assert resp.status == 200
         return await resp.text()

 async def main():
     async with aiohttp.ClientSession() as client:
         html = await fetch(client)
         print(html.decode())

 if __name__ == "__main__":
     asyncio.run(main())

[bulletmark]

@GM-Script-Writer-62850 @Carglglz After removing that ssl parameter that new version does work on MicroPython and the requests are correctly done in parallel.

However, after failing with that earlier I went back to the standard uaiohttpclientfrom micropython-lib which had not worked for me and I discovered a simple bug so I am back to using that. I submitted a PR with that fix.

[bulletmark]

@GM-Script-Writer-62850 BTW, you could actually remove the async and await from each of the 6 wrapper functions at the bottom to make it slightly more efficient. Those stubs would simply then return the coroutine directly to be awaited by the caller.

[jonnor]

There is now a quite full featured HTTP client in aiohttp. I believe that serves the purpose discussed here. And that the existing requests library does not need to be asyncified, considering that there is another library.

[GM-Script-Writer-62850]

Note that aiohttp is not shipped with: MicroPython v1.24.0-preview.224.g6c3dc0c0b on 2024-08-22; Raspberry Pi Pico W with RP2040

help('modules')
__main__          asyncio/__init__  hashlib           rp2
_asyncio          asyncio/core      heapq             select
_boot             asyncio/event     io                socket
_boot_fat         asyncio/funcs     json              ssl
_onewire          asyncio/lock      lwip              struct
_rp2              asyncio/stream    machine           sys
_thread           binascii          math              time
_webrepl          bluetooth         micropython       tls
aioble/__init__   builtins          mip/__init__      uasyncio
aioble/central    cmath             neopixel          uctypes
aioble/client     collections       network           urequests
aioble/core       cryptolib         ntptime           vfs
aioble/device     deflate           onewire           webrepl
aioble/l2cap      dht               os                webrepl_setup
aioble/peripheral ds18x20           platform          websocket
aioble/security   errno             random
aioble/server     framebuf          re
array             gc                requests/__init__
Plus any modules on the filesystem

[mattytrentini]

aiohttp is in micropython-lib and can be installed with mip: > mpremote mip install aiohttp.