Not always detecting closed connections on Fly.io

[thewilkybarkid]

This follows on from: https://community.fly.io/t/upstash-redis-and-could-not-send-http-request-to-instance-connection-error-timed-out-logs/10104

The Redis service on Fly.io (provided by Upstash) closed idled connections after an hour, but ioredis didn't always recognise it. (They increased it to a day to reduce the number of errors. I've added a ping to our app's health check, which keeps the connection alive.)

I added logging for the various events ioredis emits and could see that sometimes it would recognise the connection had closed and reconnect, but sometimes it didn't. When the connection was then used, any command would just time out.

There are a few things here:

• ioredis doesn't always recognise closed connections.
• A 'command timeout event would be useful (for logging).
• A regular ping setting would be useful to keep connections alive.

[luin]

Thanks for reaching out! It would be helpful if we can reproduce the issue locally. Not sure if the keepAlive option works for the issue.

A regular ping setting would be useful to keep connections alive.

It's easy to implement this in app code so that's a major reason why it's not built in.

[jasonkuhrt]

@luin what is the documentation for keepAlive?

[luin]

It's the native one supported by Node.js: https://nodejs.org/api/net.html