momon
commented
2 years ago I found the issue, in case anyone comes around the same, apparently the service name was changed at some point. I was using: sqlproxy-gcloud-sqlproxy.sqlproxy:port and it is now sqlproxy-gcloud-sqlproxy:port
I found the reference in the notes at the bottom here: https://github.com/rimusz/charts/blob/master/stable/gcloud-sqlproxy/README.md#upgrading
Is this a request for help?:
Yes
Version of Helm and Kubernetes:
Helm v3.9.0 Kubernetes: 1.21.10-gke.2000
Which chart:
sqlproxy
What happened:
None of my pods are able to connect to CloudSQL (mysql) anymore. It was working fine for well over a year, it suddenly broke.
What you expected to happen:
All deployments are up and the sqlproxy service is not outputting any errors, but none of the pods are connecting. If I try to access any endpoint from these pods which are not triggering the DB connection, they work. Only DB connection endpoints are broken. Also tried to login through PHPmyadmin to the DB and it wont login (because it is unable to connect to the DB).
How to reproduce it (as minimally and precisely as possible):
I am unsure, this was working perfectly fine for well over a year.
Anything else we need to know:
Yes, thinking I might have had a broken install or perhaps missing an update I tried upgrading by using:
However I am receiving: zsh: no matches found: cloudsql.instances[0].instance=instancename
Yet the instance is certainly there and the name is correct.
So I created a completely new project from scratch and created a new SQL Cloud Instance and a new Cluster and tried to rerun the setup from scratch and I am coming up with the exact same error. So I am unclear if maybe something has changed somewhere between GKE, SQL Cloud, Helm or ProxySQL which has things in a broken state.
I'm in a bit of panic as some services in production are offline as a consequence.
Appreciate the help.
UPDATE: I was actually able to run the update eventually after realizing I just needed to escape brakets (sorry first time on macos). After updating it would appear that sqlproxy is able to connect, my apps are still not connecting but reading the log from sqlproxy appears to suggest it is connected, so I will re-run some more tests and report back soon.