search

omniscale / imposm3

Imposm imports OpenStreetMap data into PostGIS
http://imposm.org/docs/imposm3/latest/
Apache License 2.0
719 stars 157 forks source link

import fails with illegal tag 0 #141

Closed AdamGlass closed 7 years ago

AdamGlass commented 7 years ago

Context

./imposm3/imposm3 import -mapping mapping.yml -read washington-latest.osm.pbf -write -connection postgis://osm:osm@localhost/osm -overwritecache -srid 4326 -deployproduction

(using 12/16/2016 and 1/19/2017 binaries on washington-latest.pbf retrieved from http://download.geofabrik.de/north-america/us/washington-latest.osm.pbf

Expected Behavior

Should import fine, has imported fine in the past. mapping.yml.gz

Actual Behavior

[Feb 10 10:33:15] [INFO] removing existing cache /tmp/imposm3 [Feb 10 10:33:16] [ 0s] C: 0/s 0/s (768000) N: 0/s 0/[Feb 10 10:33:16] [ 1s] C: 4165000/s 2115000/s (1832000) N: 36400/s 17400[Feb 10 10:33:17] [ 1s] C: 2790000/s 1586000/s (2632000) N: 37100/s 37700[Feb 10 10:33:17] [ 2s] C: 2392000/s 1690000/s (3464000) N: 59200/s 103500[Feb 10 10:33:18] [ 2s] C: 2334000/s 2127000/s (4528000) N: 51100/s 26800[Feb 10 10:33:18] [ 3s] C: 2144000/s 1408000/s (5232000) N: 54600/s 68400[Feb 10 10:33:19] [ 3s] C: 2016000/s 1391000/s (5928000) N: 58900/s 79600[Feb 10 10:33:19] [ 4s] C: 2027000/s 2095000/s (6976000) N: 54700/s 30300[Feb 10 10:33:20] [ 4s] C: 2050000/s 2207000/s (8080000) N: 55900/s 63800[Feb 10 10:33:20] [ 5s] C: 2017000/s 1761000/s (8960000) N: 59900/s 91400[Feb 10 10:33:21] [ 5s] C: 2014000/s 1981000/s (9952000) N: 59900/s 60500[Feb 10 10:33:21] [ 6s] C: 2024000/s 2128000/s (11016000) N: 59700/s 5690[Feb 10 10:33:22] [ 6s] C: 2044000/s 2252000/s (12144000) N: 58200/s 4220[Feb 10 10:33:22] [ 7s] C: 1932000/s 597000/s (12449624) N: 54700/s 1350[Feb 10 10:33:23] [ 7s] C: 1790000/s 0/s (12449624) N: 50700/s [Feb 10 10:33:23] [ 8s] C: 1790000/s 0/s (12449624) N: 50700/s [Feb 10 10:33:24] [ 8s] C: 1790000/s 0/s (12449624) N: 50700/s [Feb 10 10:33:24] [ 9s] C: 1790000/s 0/s (12449624) N: 50700/s [Feb 10 10:33:25] [ 9s] C: 1790000/s 0/s (12449624) N: 50700/s [Feb 10 10:33:25] [ 10s] C: 1790000/s 0/s (12449624) N: 50700/s [Feb 10 10:33:26] [ 10s] C: 1790000/s 0/s (12449624) N: 50700/s [Feb 10 10:33:26] [ 11s] C: 1790000/s 0/s (12449624) N: 50700/s [Feb 10 10:33:27] [ 11s] C: 1790000/s 0/s (12449624) N: 50700/s [Feb 10 10:33:27] [ 12s] C: 1790000/s 0/s (12449624) N: 50700/s [Feb 10 10:33:28] [ 12s] C: 1790000/s 0/s (12449624) N: 50700/s [Feb 10 10:33:28] [ 13s] C: 1790000/s 0/s (12449624) N: 50700/s [Feb 10 10:33:29] [ 13s] C: 1790000/s 0/s (12449624) N: 50700/s 2017/02/10 10:33:29 lowlevel.go:207: unmarshaling error (header): proto: osmpbf.BlobHeader: illegal tag 0 (wire type 6)

Possible Fix

Steps to Reproduce

  1. wget -c http://download.geofabrik.de/north-america/us/washington-latest.osm.pbf
  2. ./imposm3/imposm3 import -mapping mapping.yml -read washington-latest.osm.pbf -write -connection postgis://osm:osm@localhost/osm -overwritecache -srid 4326 -deployproduction

To be clear this used to work a few days ago. I did move to a newer washington-latest pbf recently and these issues have come up since then.

Context

Your Environment

Ubuntu 16.04 Thinkpad X1 Carbon laptop

AdamGlass commented 7 years ago

User error. My script used 'wget -c' which will do partial updates of files based on length. This resulted in corrupting the latter portion of the pbfs. My bad.