search

GoogleCloudPlatform / professional-services-data-validator

Utility to compare data between homogeneous or heterogeneous environments to ensure source and target tables match
Apache License 2.0
394 stars 110 forks source link

SSL enabled database Connectivity Option: For connecting Google CloudSQL(postgres) using SSL #1072

Closed nkonangi closed 1 month ago

nkonangi commented 8 months ago

As of now, there is no support for connecting to SSL enabled Google Cloudsql(Postgres) database. Request to enhance / add SSLenabled cloudsql database connectivity by adding 2 more optional parameters "ssl-cert", "ssl-key" to connection command of DVT.

helensilva14 commented 8 months ago

Reference links for further implementation

  1. Connect to Postgres using ssl keys, sqlalchemy+psycopg2: https://gist.github.com/tomaszjonak/c3b6ef788b4c6b0e2142d01879fac6b9

  2. psycopg2 Cheatsheet: https://gist.github.com/pfigue/3440e2bc986550a6b8ec#connect

  3. https://docs.sqlalchemy.org/en/20/dialects/postgresql.html#ssl-connections

helensilva14 commented 8 months ago

Hi @nkonangi! Thanks a lot for opening this issue, is this currently blocking your DVT usage?

nkonangi commented 8 months ago

Hi @helensilva14 , Thanks for your response. Yes, this is currently a blocker for me to adopt DVT. What you shared earlier are the generic python code solutions. But I am looking for a way to pass these additional SSL parameters to data-validations connection command. Where DVT will take care of internally passing SSL details to establish connectivity. Please let me know if that clarifes your question.

helensilva14 commented 7 months ago

Hi @nkonangi, your answer did provide more context, thanks a lot! We'll keep analyzing this feature request.

For our developer team: @nj1973 and @nehanene15 could you PTAL at this issue and add your thoughts?

nkonangi commented 7 months ago

thank you for looking into it.

GabrieleCalarota commented 2 months ago

Also running into this issue when data-validating with mutual SSL authentication on both ends.

Happy to contribute to make this happen! 🚀

nehanene15 commented 2 months ago

@GabrieleCalarota if you are able to contribute that would be great! It will be easier since you have a SSL enabled DB to test on.

This is where we build the SQLalchemy URL to connect. Seems like we should be able to add SSL connection parameters as shown here.

This is where you would add the --ssl-cert and --ssl-key flags for the PG connection command.

nj1973 commented 1 month ago

Change of plan, we've decided it would be better to implement this via documenting existing PostgreSQL support of environment variables and not the addition of new options.