Cannot export resulting .pbf into postgres with osm2pgsql: invalid BlobHeader size

[stefda]

After successfully generating a db with water polygons I cannot export the data into postgres via osm2pgsql. Instead I get "PBF error: invalid BlobHeader size (> max_blob_header_size)".

osmcoastline version 2.1.4 osm2pgsql version 0.94.0 (64 bit id space)

The db is generated with

osmcoastline --srs=3857 --output-polygons=water --output-lines --verbose -o final.osm.pbf filtered.osm.pbf

The export command is osm2pgsql -d osm -H <host> -U <username> -W final.osm.pbf and the full output is

Using built-in tag processing pipeline
Using projection SRS 3857 (Spherical Mercator)
Setting up table: planet_osm_point
Setting up table: planet_osm_line
Setting up table: planet_osm_polygon
Setting up table: planet_osm_roads
Allocating memory for dense node cache
Allocating dense node cache in one big chunk
Allocating memory for sparse node cache
Sharing dense sparse
Node-cache: cache=800MB, maxblocks=12800*65536, allocation method=3

Reading in file: final.osm.pbf
Using PBF parser.
node cache: stored: 0(-nan%), storage efficiency: -nan% (dense blocks: 0, sparse nodes: 0), hit rate: -nan%
Osm2pgsql failed due to ERROR: PBF error: invalid BlobHeader size (> max_blob_header_size)

Where did I make a mistake?

[stefda]

As a side note, the extracted data imports fine into QGIS.

I realise this is probably a osm2pgsql "issue" that has to do with the upper limit on the size of the data set. Perhaps someone here can point me to an alternative way for importing the extract into postgres.

[stefda]

I also tried osmosis which errs in a similar way:

SEVERE: Thread for task 1-read-pbf failed
org.openstreetmap.osmosis.core.OsmosisRuntimeException: Unable to process PBF stream
        at crosby.binary.osmosis.OsmosisReader.run(OsmosisReader.java:48)
        at java.lang.Thread.run(Thread.java:748)
Caused by: org.openstreetmap.osmosis.osmbinary.file.FileFormatException: Unexpectedly long header 65536 bytes. Possibly corrupt file.
        at org.openstreetmap.osmosis.osmbinary.file.FileBlockHead.readHead(FileBlockHead.java:50)
        at org.openstreetmap.osmosis.osmbinary.file.FileBlock.process(FileBlock.java:130)
        at org.openstreetmap.osmosis.osmbinary.file.BlockInputStream.process(BlockInputStream.java:34)
        at crosby.binary.osmosis.OsmosisReader.run(OsmosisReader.java:45)

[stefda]

All right, ogr2org imports the data into postgres fine. I guess this can be closed but I'd still appreciate if someone could shed some light onto why osm2pgsql and osmosis are unhappy about the header size.

[joto]

The file created by osmcoastline is not an OSM PBF file, but a spatialite database file. So osm2pgsql or any other tool expecting an OSM PBF file can't read it.

[stefda]

Oh, OK, thanks for the insight!