search

mojodna / osm-pds-pipelines

OSM PDS pipeline
https://quay.io/repository/mojodna/osm-pds-pipelines
ISC License
31 stars 4 forks source link

planet-230508.osm.pbf is corrupted #46

Closed erik closed 1 year ago

erik commented 1 year ago

Hi, I noticed a process consuming PBF planet files in osm-pds failed today. Seems the most recent upload was truncated. planet-230508.osm.pbf is 69GB on planet.openstreetmap.org, but only 20GB in s3://osm-pds.

Checksum also doesn't match:

$ aws s3 cp s3://osm-pds/2023/planet-230508.osm.pbf.md5 -
7868fa07e02c416362cb636a866ad2be  planet-230508.osm.pbf
$ aws s3 cp s3://osm-pds/2023/planet-230508.osm.pbf - | md5sum
7df779424c32e2dc5cb2c837a6c98867  -

Prior week looks fine: 

$ aws s3 cp s3://osm-pds/2023/planet-230501.osm.pbf.md5 -
28eff62ccb648e73defa6223500d25f1  planet-230501.osm.pbf
$ aws s3 cp s3://osm-pds/2023/planet-230501.osm.pbf - | md5sum
28eff62ccb648e73defa6223500d25f1  -
mojodna commented 1 year ago

planet-230508.osm.pbf is now 68.9GB in S3. Thanks for reporting this.