search

samuong / alpaca

A local HTTP proxy for command-line tools. Supports PAC scripts and NTLM authentication.
Apache License 2.0
184 stars 31 forks source link

Too many open files #9

Closed samuong closed 5 years ago

samuong commented 5 years ago

A long-running Alpaca session ended up not being able to proxy traffic, and had the following error messages in the logs:

2019/06/03 09:15:53 server.go:2979: http: Accept error: accept tcp 127.0.0.1:3128: accept: too many open files; retrying in 5ms
2019/06/03 09:15:55 server.go:2979: http: Accept error: accept tcp 127.0.0.1:3128: accept: too many open files; retrying in 10ms
2019/06/03 09:15:55 server.go:2979: http: Accept error: accept tcp 127.0.0.1:3128: accept: too many open files; retrying in 20ms
2019/06/03 09:15:55 server.go:2979: http: Accept error: accept tcp 127.0.0.1:3128: accept: too many open files; retrying in 40ms
2019/06/03 09:15:55 server.go:2979: http: Accept error: accept tcp 127.0.0.1:3128: accept: too many open files; retrying in 80ms

It looks like Alpaca is leaking connections, maybe due to not closing one half of the CONNECT requests (ref ProxyHandler.handleConnect).

In addition to fixing the leak, there should be better visibility into what's going on in terms of open connections.

samuong commented 5 years ago

This should be fixed by 0904ccc. I've been running with this change locally for a few days now, and it appears to clean up all the leaked file descriptors.

In case it's useful in the future, the number of open connections (in each direction of a CONNECT tunnel) can be found using something like lsof -Pn -c alpaca | awk '!/ESTABLISHED/{next} /:3128/{proxy++;next} {tunnel++} END{printf "%d (proxy) %d (tunnel)\n", proxy, tunnel}'.