RestComm / Restcomm-Connect

The Open Source Cloud Communications Platform
http://www.restcomm.com/
GNU Affero General Public License v3.0
242 stars 215 forks source link

MySQL connections don't recover after MySQL restart, with single DB host #2856

Open peteroyle opened 6 years ago

peteroyle commented 6 years ago

/kind bug

What happened: When using MySQL as the database, with just a single MySQL host (so MYSQL_SNDHOST=''), if the MySQL server is restarted for maintenance, all calls to RestComm will fail until RestComm itself is restarted.

What you expected to happen: RestComm should gracefully recover the disconnected MySQL connections in the pool so that eventually calls can continue without manual intervention.

How to reproduce it (as minimally and precisely as possible): Start up restcomm with a primary MySQL host configured, and MYSQL_SNDHOST=''. Restart MySQL. Make a call to RestComm and observe that the call does not connect, and a stack trace is produced with a message such as "Can not read response from server. Expected to read 4 bytes, read 0 bytes before connection was unexpectedly lost"

Continue trying to make calls and note that the situation never improves until you restart RestComm.

Anything else we need to know?: There is already the necessary configuration to fix this in config-mysql.sh, but it's only ever applied if MYSQL_SNDHOST is a non-empty string. I am about to create a pull request which basically copies the missing configuration to the "datasource" element which is applied when MYSQL_SNDHOST=''.

I'm not sure why there is a difference there, maybe there's a reason for it that I'm missing?

Environment:

gsaslis commented 6 years ago

Thanks a lot @peteroyle for the complete and well-documented issue. <3

@leftyb can you please take a look at this one?