contribsys / faktory

Language-agnostic persistent background job server
https://contribsys.com/faktory/
Other
5.73k stars 227 forks source link

Question about concurrency vs connections used #358

Closed mivok closed 3 years ago

mivok commented 3 years ago

Hi. We currently have several faktory go workers set to use a concurrency of 64, and a connection pool size of 2 more than the concurrency (66 in this case). It's created with faktory.NewPool. With this setting we are running netstat and see 70-90 established connections to the faktory server, and not the roughly 64 that we would expect. We are curious why the behavior would be such that more connections are used than the amount of concurrent workers, and why we seem to have more connections than the pool size.

Here's the netstat command we used (and if I don't add the wc -l and look at the output manually I see that the grep is only catching the appropriate connections - e.g. there is nothing using a client port of 7419).

$ netstat -nt | grep :7419 | grep ESTABLISHED | wc -l
83
mperham commented 3 years ago

I have no idea why it might have more connections, no one's reported this before. You'd need to debug your app and track where each of those connections comes from. Please let me know what you learn.