anthonyfok
commented
3 years ago Hi @jvanulde,
This first version of "GPKG to PostgreSQL dump" GitHub action/workflow is ready for review.
Some notes:
- I changed the name the PostgreSQL custom-format archive file to "opendrr-boundaries.dump" to distinguish it from human-readable SQL script
- As an experiment, instead of commiting back to the GitHub repo, opendrr-boundaries.dump is uploaded to a public "OpenDRR" bucket on Backblaze B2.
- There is $0.00 egress fee due to the awesome partnership between Backblaze and Cloudflare:
- https://www.backblaze.com/blog/backblaze-and-cloudflare-partner-to-provide-free-data-transfer/
- https://help.backblaze.com/hc/en-us/articles/217666928-Using-Backblaze-B2-with-the-Cloudflare-CDN
- The backstory from Brian Wilson, CTO of Backblaze, explaining why they are doing this: https://www.reddit.com/r/backblaze/comments/l2lq42/how_long_will_cloudflare_peering_stay_for/
- The file is available for download ($0 fee) from https://opendrr.eccp.ca/file/OpenDRR/opendrr-boundaries.dump , where opendrr.eccp.ca is subdomain linked with Cloudflare CDN (internally, a CNAME to the original Backblaze server)
The opendrr-boundaries.dump at 822MB comes out smaller than the existing opendrr-boundaries.sql at 835MB. I haven't investigated the cause. Perhaps it is due to the recent change in GPKG files? Or better compression? Or due to different parameters that I used?
Some ideas for future revisions:
- Find ways to show that the database archive stored on B2 or S3 matches the repo (using e.g. timestamp, checksum, commit hash, GPG signature?)
- Find ways to verify that the database archive is identical to the database content
- Provide some record or metadata on how the database archive was generated (on which OS, which version of GDAL, PostgreSQL, etc.)
Thanks!
Fixes #10