wasmerio / winterjs

Winter is coming... ❄️
https://winterjs.org/
MIT License
3.05k stars 53 forks source link

Too many open files on native #45

Open theduke opened 9 months ago

theduke commented 9 months ago

Right now we are reaching "too many open files" when running wrk and winterjs natively:

$ winterjs ./simple.js

And, in other shell tab:

$ wrk -t12 -c400 -d10s http://127.0.0.1:8080 Running 10s test @ http://127.0.0.1:8080 12 threads and 400 connections Thread Stats Avg Stdev Max +/- Stdev Latency 3.91ms 10.03ms 294.68ms 99.16% Req/Sec 6.25k 2.00k 11.33k 73.92% 747990 requests in 10.02s, 122.69MB read Socket errors: connect 155, read 306, write 0, timeout 0 Requests/sec: 74615.22 Transfer/sec: 12.24MB

I do have ulimit -n of 256. Once I did update ulimit -n 10000 things were solved. However, why are we opening that many files? We should fix this.

https://linear.app/wasmer/issue/WAX-86/too-many-open-files-winterjs

wagk commented 8 months ago

I suspect that this is not winterjs's fault, but wrk, in that wrk is probably keeping each connection open for the duration of the benchmark (so in this case up to 400 of them) and quickly depletes the file handles available.

As a quick test I ran it with -n100 (and ulimit -n 256) and no errors were emitted. A quick glance over winterjs code also does not raise any obvious issues relating to holding on to unusual amounts of file handles.