fuelkoy
commented
7 months ago This can be tackled by lazy init a global pool and connector. As pointed out by @jackwotherspoon
One thing you could try to improve performance would be to lazy init a global pool and connector. This way it is initialized during the first request when you have access to the credentials but then saved for subsequent requests. This is actually mentioned as a tip for Cloud Functions. The only caveat being that you would probably also want to turn on "Always on CPU" so that the Cloud SQL Node Connector's background refreshes have access to CPU and are not throttled (we will be implementing a lazy refresh feature soon so that this is no longer required https://github.com/GoogleCloudPlatform/cloud-sql-nodejs-connector/issues/285)
https://github.com/GoogleCloudPlatform/cloud-sql-nodejs-connector/issues/312#issuecomment-2009503838
I will now close this issue as solved.
[REQUIRED] Environment info
firebase-tools: 13.5.1
Platform:
[REQUIRED] Test case
https://github.com/fuelkoy/nodeJsSqlConnector
[REQUIRED] Steps to reproduce
Set the following in tsconfig.json to be able to use ESM with minimal errors
pnpm i @google-cloud/cloud-sql-connector @google-cloud/functions-framework (I'm using pnpm) dotenv google-auth-library mysql2 esbuild npm-run-all[REQUIRED] Expected behavior
Firebase functions should pick up GOOGLE_APPLICATION_CREDENTIALS even for cases it is needed outside the functions (like the case for the Cloud SQL Node Connector)
Firebase should populate ADC automatically (as when starting the emulator) or pick it up if
gcloud initandgcloud auth application-default loginare called. GOOGLE_APPLICATION_CREDENTIALS points toC:\Users\MyUser\AppData\Roaming\firebase\ownerEmail_application_default_credentials.jsonwhich is created automatically when starting the emulator. But when developing locally trying to use ADC from gcloud auth application-default login or deploying GOOGLE_APPLICATION_CREDENTIALS is empty and initializing the connector and the pool on global scope will fail with the following error:Error: Could not load the default credentials.Cloud SQL Node Connector documentation shows that the connector and pool are made globally: https://github.com/GoogleCloudPlatform/cloud-sql-nodejs-connector?tab=readme-ov-file#using-with-mysqlIt is possible to workaround this by making a initializeConnection function that will then be called inside each funtion where pool and connector are gotten and then used. BUT this will introduce a hefty 500–930 ms connection initialization penalty to each function start
Firebase Functions should pick up ADC also when deploying from
C:\Users\MyUser\AppData\Roaming\firebase\ownerEmail_application_default_credentials.json$HOME/.config/gcloud/application_default_credentials.jsonWindows:%APPDATA%\gcloud\application_default_credentials.jsonhttps://cloud.google.com/docs/authentication/application-default-credentials#personalI had a long discussion with the Cloud SQL Node Connector team about this (found in here: https://github.com/GoogleCloudPlatform/cloud-sql-nodejs-connector/issues/312#issuecomment-2008879612) and it points out to the problem of setting the GOOGLE_APPLICATION_CREDENTIALS in Firebase Functions (windows specifically). The problem seems to relate to this comment:
https://github.com/firebase/firebase-tools/issues/6499#issuecomment-1824982590
If this is the case and Firebase Functions will not pick up GOOGLE_APPLICATION_CREDENTIALS in global scope Firebase Functions with the Cloud SQL with the Cloud SQL Node Connector (which Google itself recommends to use over Cloud SQL Auth proxy https://cloud.google.com/blog/products/databases/cloud-sql-nodejs-connector-is-ga) are non-usable due to the non-accepted performance penalty.
[REQUIRED] Actual behavior
Run to use gcloud auth application-default login ADC:
$env:GOOGLE_APPLICATION_CREDENTIALS="C:\Users\Kehitys\AppData\Roaming\gcloud\application_default_credentials.json"pnpm dev (build and start emulator):
pnpm deploy:
I hope you can help with this issue, as this makes our company unable to use cloud SQL and would require us to move to a different managed database provider, which we do not prefer. If you need any more information, I'm happy to help. Thank you in advance for all the help; it is greatly appreciated.