search

steadyequipment / node-firestore-backup

Google Firebase Firestore backup tool
190 stars 51 forks source link

Support creating a compressed archive directly #32

Open lewchuk opened 6 years ago

lewchuk commented 6 years ago

The JSON files created by backup are highly compressible, especially if documents in a collection have similar fields. For my data a basic zip got ~70% compression. It would be great if the backup supported writing directly to a compressed archive to limit intermediate disk space.

yoiang commented 6 years ago

Thinking out loud this makes sense to include in the tool vs recommending in people's workflow if it's a common usage we want to encourage. It makes complete sense to me, does anyone have a suggestion why this wouldn't?

atlanteh commented 6 years ago

Since this is a tool used mainly in scripts, I'd say it might be better to allow piping the data to another program rather than creating a specific zip with specific properties that people would probably like to change. Please note that, if the DB size is bigger than available memory, then I'm not sure piping will work (and also zipping for that matter).

atlanteh commented 6 years ago

Since I was just told this tool uses multiple files, I think that piping will be a bit harder, so maybe the zip should be good enough.