search

xsxo / fiberhttp

High Performance HTTP Requests Library
Apache License 2.0
11 stars 1 forks source link

fiberhttp doesn't work with FastAPI #3

Open NostraDavid opened 1 month ago

NostraDavid commented 1 month ago

Was looking at your benchmarks, and noticed you implemented your own server, which is a bit of a mess, because I'm pretty sure some connection weren't being closed by fiberhttp. Anyway, that's beside the point.

I was super curious what would happen if I replaced your server with FastAPI (supposedly the fastest Pythonic HTTP server), and while the other benchmark libs worked fine, fiberhttp breaks because it closes the connection after the first GET.

I thought it may be the async nature of FastAPI, so I changed it to sync, but alas... I then tried Flask (which is a bit slower (about 50%-ish)) and that worked fine.

Anyway, I figured you'd want to take a look at my code: https://github.com/NostraDavid/fiberhttp/tree/polish-benchmarks

Check out the benchmarks2 folder (initially I tried rewriting your code, but ran into walls, so I figured starting from scratch).

Do note I'm using uv so I don't have to manage my Python versions (I can just set it in my script), but uv run server_fastapi.py and then uv run benchmark.py should work (presuming you've installed uv - I believe you're using Windows?)

Anyway, I figure you'd want to know. If you want to know anything else let me know.

NostraDavid commented 1 month ago

PS: I found your lib via this reddit thread :D

I was writing an ingestion (context: I'm a data engineer) to download data from https://opendata.dwd.de (it's both an FTP server - no not SFTP; an actual FTP server - and a regular HTTP(S) server) and I was searching for the fastest HTTP lib.

For now I ended up using urllib3 with a ThreadPoolExecutor to fill my 1Gbit connection :D

xsxo commented 3 weeks ago

Hi I have never used FastAPI before, and I know it’s simple, but I don't want to dive into it at this time.

I also added a server written in C and another server written in GoLang to achieve high performance and fairness between the libraries.

Thank you so much for your contribution. Did you notice any issues while using the library?

xsxo commented 3 weeks ago

PS: I found your lib via this reddit thread :D

I was writing an ingestion (context: I'm a data engineer) to download data from https://opendata.dwd.de (it's both an FTP server - no not SFTP; an actual FTP server - and a regular HTTP(S) server) and I was searching for the fastest HTTP lib.

For now I ended up using urllib3 with a ThreadPoolExecutor to fill my 1Gbit connection :D

wow thats good, nice to meet you