Closing connections to peers

[knazarov]

I'm starting a timer in init_worker_by_lua_block() and checking upstream peer health via http. If a peer is not healthy, I set its 'down' flag to 1. Which routes new connections to other peers. However, old connections are not closed and clients keep using the non-healthy peers.

Could you please point me to any examples on how I can close existing connections to one of the peers from the timer handler?

[agentzh]

@racktear I'm not sure I understand what you are talking about given so little info. Maybe you should just debug your own Lua code for your own issues?

[knazarov]

Let me clarify.

I have one upstream with 2 peers and I'm implementing a custom health check of those peers. The health check works in a timer handler, and makes periodic http requests to both of those peers to determine whether they are healthy. Then using the result of those requests, I'm marking the peers as either healthy or not healthy. The incoming traffic will go to one of the healthy peers.

To do that I've ported the implementation of set_peer_down from lua-upstream-nginx-module to support streams. Initially I expected that if I set the 'down' flag on the peer, nginx will not route new connections to that peer, and it will close existing connections to it. The latter appears to be false. Which is not a bug at all, but just an implementation detail of nginx.

Now I'm just looking for a way to somehow in addition close all existing connections to the peer when I detect that it's not healthy. Do you know if there is a way to do so from the timer handler?

[agentzh]

@racktear No, it should be your request handlers closing the connections themselves. You cannot do that directly from within the timer handler, which is unsafe per se. Your timer handler should only notify those affected request handlers in some way.

[knazarov]

@agentzh my problem is that I don't serve content by lua blocks. I just use proxy_pass.

So looks like I'm out of luck with pure lua solutions :(

[agentzh]

@racktear Actually you can do that with pure lua solution. The problem is that you are not using a pure Lua solution for proxying.

[knazarov]

Answering my own question. Just in case people google for this. There is really no easy way to force close the upstream connection from lua. But I've made a patch to ngx_stream_proxy_module.c that checks for the 'down' flag in the data handler, ngx_stream_proxy_process(). If the flag is set, the proxy process terminates the connection gracefully.

Here's the patch: https://gist.github.com/racktear/4be8934f015d523aa85dcdca47478241

This way one can manage stream proxying from lua, relying on nginx upstream mechanisms.