search

urllib3 / urllib3

urllib3 is a user-friendly HTTP client library for Python
https://urllib3.readthedocs.io
MIT License
3.71k stars 1.12k forks source link

NodeJS + pyodide support #3400

Open joemarshall opened 3 weeks ago

joemarshall commented 3 weeks ago

Context

Recently released Pyodide 0.26.0 supports webassembly Javascript Promise Integration. This means it is practical to do synchronous http requests in webassembly python using the standard (asynchronous) node-fetch api.

This would make urllib3 work in webassembly on node.js supporting platforms such as Cloudflare workers.

Once support for JSPI is released generally in browsers, it would be possible to unify the webassembly support to use this, which removes the need for web-workers for streaming http requests and would simplify the emscripten backend.

Alternatives

It is possible to hack in node support using the node xmlhttprequest module, but the way that synchronous requests are handled is gross - it has to launch processes to handle them, and it is not a usable hack for server side python code.

Duplicate

I don't think so. I think I mentioned it as future work when making the pyodide browser support.

Contribution

I'm contracted to do this work already, and would like to upstream it once it is working.

sethmlarson commented 3 weeks ago

Hey @joemarshall, nice to see you again! :) Happy to accept patches which makes urllib3 work better in those environments. If it's a big simplification (and based on your judgement) I'd be okay removing support for the more complicated backend if the simpler one is the way forwards in your mind. Let me know your thoughts on this, thanks again!