Open willm opened 7 years ago
Hello Will, this is a known issue with the Kubernetes API server behavior. It definitely deserves a note in the documentation, though, so thanks for bringing it back to my attention! If there is no client activity on a socket, then the server will close the connection eventually, even if no timeout is set for the connection. This unfortunately also appears to apply to watch endpoints as well.
In effect there is an undocumented minimum timeout on all watch sockets.
Our internal solution was to set a moderate timeout on the watch socket and continually reopen it unless there is an error (other than the expected timeout error, of course). We also record the last received version number to prevent loss of events during the reconnect period. You will always get an end
event eventually, so we plan around it for long-running listeners.
I may consider writing a wrapper to abstract this in the future, but that isn't in my immediate plans. Pull requests are more than welcome if you want to take a stab at it, though.
@ryclarke I've had a go at a fix, but I'm not sure how to set up my environment so the existing test suite runs. Could you check when you have a moment please?
It seems I may have made the problem worse, please don't merge this, I'll look in more detail
Hi, I'm using the watch api in my application. I've noticed that every so often, the
'end'
event gets fired for the request to the watch endpoint. Since no event listener is registered for this event, the watcher effectively silently stops listening for changes. I'm wondering what the preferred behaviour is here? We could just log a warning call thestart()
function to restart the service watcher. I can make a pull request to fix this, just wondered what people's thoughts are first.