search

rwnx / pynonymizer

A universal tool for translating sensitive production database dumps into anonymized copies.
https://pypi.org/project/pynonymizer/
MIT License
103 stars 38 forks source link

Additional Mysql connection parameters #38

Closed nladuguie closed 3 years ago

nladuguie commented 3 years ago

Hi,

I would like to use Pynonymizer tool connecting to AWS RDS Mysql instances. But for Mysql RDS connexion, we require to set Mysql SSL connexion parameters. Currently, pynonymizer provides a way to provide Mysql dump additional parameters via "additional_opts" and "dump_opts" (in MySqlDumpRunner). But for Mysql queries execution, it's not possible to provide any additional options, so that we can configure the SSL for Mysql connection. May be you could in the class MySqlCmdRunner the same additional options as in MySqlDumpRunner class (https://github.com/jerometwell/pynonymizer/blob/02593b18e6f2ae3b173689d518e138d002551e03/pynonymizer/database/mysql/execution.py#L16).

rwnx commented 3 years ago

Hi, I would like to support this properly and allow some kind of unrestricted access to the underlying dump/cmd CLIs, as there's clearly a complex list of requirements that i don't want to re-implement 😇 I don't have an estimate for you right now but i'll keep this issue updated with any progress.

As a workaround, the subprocess call should copy in the current environment, so you might be able to set options in the underlying mysql call by setting the environment vars or default config files like my.cnf- might be something to explore if this is super time-critical.

nladuguie commented 3 years ago

@jerometwell yes that was my plan to add my.cnf file, waiting for your implementation. Thanks for your support, and congrats because your tool is really nice.

rwnx commented 3 years ago

Oops, I'm not sure Closing this issue is what i want. I've just merged this feature. If you can test on master, that would be great, otherwise This should go out with v1.17.0 over the next few days.

rwnx commented 3 years ago

pynonymizer v1.17.0 has been released, please check and report any issues 😇 !

https://pypi.org/project/pynonymizer/1.17.0/

daaain commented 3 years ago

I had to set up custom SSL with Postgres and this didn't work (perhaps unsurprisingly):

additional_opts = "sslmode=verify-ca sslrootcert=/certs/server-ca.pem sslcert=/certs/client-cert.pem sslkey=/certs/client-key.pem"

But I managed with environment variables:

            - name: PGSSLMODE
              value: "verify-ca"
            - name: PGSSLCERT
              value: "/certs/client-cert.pem"
            - name: PGSSLKEY
              value: "/certs/client-key.pem"
            - name: PGSSLROOTCERT
              value: "/certs/server-ca.pem"

Not sure if this solves the issue with MySQL but thought I should share as this is the only Github issue about SSL.