search

omniscale / imposm3

Imposm imports OpenStreetMap data into PostGIS
http://imposm.org/docs/imposm3/latest/
Apache License 2.0
711 stars 156 forks source link

import read fails with unmarshaling error "wiretype end group for non-group" #187

Closed sgrossberndt closed 5 years ago

sgrossberndt commented 5 years ago

Context

Import read fails with unmarshaling error "wiretype end group for non-group" for http://download.geofabrik.de/europe/germany/niedersachsen-latest.osm.pbf (available for one week from tomorrow as http://download.geofabrik.de/europe/germany/niedersachsen-190121.osm.pbf

Error comes from https://github.com/omniscale/imposm3/blob/master/parser/pbf/internal/osmpbf/osmformat.pb.go#L2745

Expected Behavior

imposm import -read download/niedersachsen-latest.osm.pbf should read the PBF-file into cache

Actual Behavior

[Jan 21 18:05:14] [INFO] [reader] reading download/niedersachsen-latest.osm.pbf with data till 2019-01-20 22:14:02 +0100 CET
[Jan 21 18:06:14] [INFO] [  1m0s] C:  596000/s (29231879) N:   15100/s (743327) W:  269300/s (3048000) R:      0/s (0)
2019/01/21 18:06:14 lowlevel.go:81: unmarshaling error: proto: StringTable: wiretype end group for non-group04000/s (3200000) R:      0/s      0/s (0)
panic: unmarshaling error: proto: StringTable: wiretype end group for non-group

goroutine 13 [running]:
log.Panic(0xc4202bfda0, 0x2, 0x2)
        /home/travis/.gimme/versions/go1.10.linux.amd64/src/log/log.go:326 +0xc0
github.com/omniscale/imposm3/parser/pbf.readPrimitiveBlock(0x7fff1deb1857, 0x25, 0x103167e2, 0xc40002c1c8, 0xc42294a000)
        /home/travis/gopath/src/github.com/omniscale/imposm3/parser/pbf/lowlevel.go:81 +0x187
github.com/omniscale/imposm3/parser/pbf.(*Parser).parseBlock(0xc420088640, 0x7fff1deb1857, 0x25, 0x103167e2, 0x2c1c8)
        /home/travis/gopath/src/github.com/omniscale/imposm3/parser/pbf/process.go:89 +0x68
github.com/omniscale/imposm3/parser/pbf.(*Parser).Parse.func1(0xc42006e900, 0xc420088640)
        /home/travis/gopath/src/github.com/omniscale/imposm3/parser/pbf/process.go:56 +0x89
created by github.com/omniscale/imposm3/parser/pbf.(*Parser).Parse
        /home/travis/gopath/src/github.com/omniscale/imposm3/parser/pbf/process.go:54 +0xcb

Steps to Reproduce

  1. Download http://download.geofabrik.de/europe/germany/niedersachsen-190121.osm.pbf
  2. imposm import -config imposm-config.json -overwritecache -read download/niedersachsen-latest.osm.pbf

Context

Importing a current state of the OSM files fails.

Your Environment

sgrossberndt commented 5 years ago

This issue happened due to an incomplete download of the PBF file which went by undetected.