rap2hpoutre / pg-anonymizer

Dump anonymized PostgreSQL database with a NodeJS CLI
https://raph.site
MIT License
223 stars 31 forks source link

params like skip not working or list with strange rules #44

Open jeremieflrnt opened 1 year ago

jeremieflrnt commented 1 year ago

The project looks cool! Thanks for sharing 👍

I tried to use it a bit, and I manage to give a list of columns. But it was working only if it was written only the column name, like --list=email and not --list=public.user.email like in the docs. And also, --skip is not skipping the table I want it to skip. This time no matter how I type it: --skip=public.videoLink, --skip=videoLink, --skip=public."videoLink"

I still see (kept relevant infos..):

npx pg-anonymizer postgresql://user:password@localhost:5432/mydb -o dump.sql --skip=public."videoLink" --list=email

List: email
Skipping: public.videolink
Output file: dump.sql

Launching pg_dump...
Command pg_dump started, running anonymization.

public."User": idclient, [email]
Anonymizing 1 column...

public."VideoLink": id, link, serviceid
Skipping... no matching columns

And it's not skipped.

jackall3n commented 1 year ago

Hi @jeremieflrnt,

But it was working only if it was written only the column name, like --list=email and not --list=public.user.email like in the docs.

For the current version, to specify columns on specific tables you'll need to include the quotes " for the tables that need them. In your instance your table is called public."User", meaning you'd have to write your column as --list public.\"user\".email. However, this is something that can change I think.

--skip is not skipping the table I want it to skip

Judging by the question and output, you're expecting --skip to not output the table into dump.sql at all, is that right?

I'll answer assuming that is what you meant. The --skip flag is used for pg-anonymizer to determine whether to skip the anonymization of a table, not to skip the output of a table entirely. As pg-anonymizer parses the output supplied by pg-dump you'll need to parse args to pg-dump. The arg to skip a table is -T [tablename]. To pass args, you simply add -- [pg-dump args] to the end of your command

An example would be:

npx pg-anonymizer postgresql://user:password@localhost:5432/mydb -o dump.sql --list=email -- -T videoLink