move pg_dump and output to streams

rap2hpoutre / pg-anonymizer

Dump anonymized PostgreSQL database with a NodeJS CLI

https://raph.site

MIT License

223 stars 31 forks source link

Closed imreACTmd closed 2 years ago

imreACTmd commented 2 years ago

pg-anonymizer doesn't require a memory buffer any more and can handle any size database.

imreACTmd commented 2 years ago

I tested it with a 400 GB dump - it took around the same time to anonymize as if I ran pg_dump by itself. Very speedy!

rap2hpoutre commented 2 years ago

Wow! Thank you so much @imreACTmd! 🙏 This is a huge step forward! I will take a look at your PR tomorrow hopefully.