search

micropython / micropython-lib

Core Python libraries ported to MicroPython
Other
2.3k stars 980 forks source link

Change HTTP/1.0 to HTTP/1.1 in `requests` Python module #861

Open sgbaird opened 1 month ago

sgbaird commented 1 month ago

On the Pico W, I'm suddenly having trouble accessing an API that used to be compatible. The issue seems to be the HTTP version, which I documented at https://github.com/orgs/micropython/discussions/15112. As somewhat of a last try, I copied over https://github.com/micropython/micropython-lib/blob/e025c843b60e93689f0f991d753010bb5bd6a722/python-ecosys/requests/requests/__init__.py into a urequests_2.py, used import urequests_2 as urequests, and simply replaced 1.0 with 1.1:

s.write(b"%s /%s HTTP/1.0\r\n" % (method, path))

with:

s.write(b"%s /%s HTTP/1.1\r\n" % (method, path))

I was able to successfully send data through the API after that. I also wanted to note that I couldn't use the version on master due to ImportError: no module named 'tls', hence the pinned version from above. I have tutorials, videos, and a module within a course dependent on this, so it would be great to have this incorporated into a stable version of Pico W MicroPython if it's feasible.

mattytrentini commented 1 month ago

I think it's a good idea to merge this as-is.

However, it would also be useful to take a look at CircuitPython's request library. It was based on this one but has since had some updates - it'd be good to evaluate if other changes should be integrated too.

dpgeorge commented 1 month ago

See related #844.

It's a shame that HTTP/1.0 is becoming obsolete because it's a nice and simple (and still functional) protocol.

But, yeah, we need to update to use HTTP/1.1. The problem is it's not as simple as changing the 1.0 to a 1.1. The code also needs to be improved to support chunked responses, at the very least. Also probably need to add Accept-Encoding: identity to the headers to prevent the server from responding with compressed data, which we can't deal with.

Note that our aiohttp library does support HTTP/1.1, but it's not really a drop-in replacement for requests.

In summary: let's update to HTTP/1.1 but it needs to be done to the spec, and tested well.