openstreetmap / osmosis

Osmosis is a command line Java application for processing OSM data.
http://wiki.openstreetmap.org/wiki/Osmosis
676 stars 207 forks source link

Cannot read .bz2 files #100

Open isheff opened 3 years ago

isheff commented 3 years ago

Bug

Osmosis attempts to read .bz2 files as gzip, and fails

Steps to reproduce

On a debian linux machine:

wget https://ftp5.gwdg.de/pub/misc/openstreetmap/planet.openstreetmap.org/planet/planet-latest.osm.bz2

check the md5 hash against https://planet.openstreetmap.org/ to ensure it's correct.

wget https://github.com/openstreetmap/osmosis/releases/download/0.48.3/osmosis-0.48.3.tgz
tar zxvf osmosis-0.48.3.tgz
bin/osmosis --read-xml file="planet-latest.osm.bz2" compressionMethod="bzip2"    --write-xml file="copy.osm.bz2" compressionMethod="bzip2"

This produces the error:

Oct 08, 2021 8:15:53 PM org.openstreetmap.osmosis.core.Osmosis run
INFO: Osmosis Version 0.48.3
Oct 08, 2021 8:15:53 PM org.openstreetmap.osmosis.core.Osmosis run
INFO: Preparing pipeline.
Oct 08, 2021 8:15:53 PM org.openstreetmap.osmosis.core.Osmosis run
INFO: Launching pipeline execution.
Oct 08, 2021 8:15:53 PM org.openstreetmap.osmosis.core.Osmosis run
INFO: Pipeline executing, waiting for completion.
Oct 08, 2021 8:15:53 PM org.openstreetmap.osmosis.core.pipeline.common.ActiveTaskManager waitForCompletion
SEVERE: Thread for task 1-read-xml failed
org.openstreetmap.osmosis.core.OsmosisRuntimeException: Unable to unzip gz file planet-latest.osm.bz2.
    at org.openstreetmap.osmosis.xml.v0_6.impl.BaseXMLReader.unzipParse(BaseXMLReader.java:109)
    at org.openstreetmap.osmosis.xml.v0_6.impl.BaseXMLReader.handleXML(BaseXMLReader.java:84)
    at org.openstreetmap.osmosis.xml.v0_6.XmlReader.run(XmlReader.java:52)
    at java.base/java.lang.Thread.run(Thread.java:829)
Caused by: java.util.zip.ZipException: Not in GZIP format
    at java.base/java.util.zip.GZIPInputStream.readHeader(GZIPInputStream.java:166)
    at java.base/java.util.zip.GZIPInputStream.<init>(GZIPInputStream.java:80)
    at java.base/java.util.zip.GZIPInputStream.<init>(GZIPInputStream.java:92)
    at org.openstreetmap.osmosis.xml.v0_6.impl.BaseXMLReader.unzipParse(BaseXMLReader.java:101)
    ... 3 more

Oct 08, 2021 8:15:53 PM org.openstreetmap.osmosis.core.Osmosis main
SEVERE: Execution aborted.
org.openstreetmap.osmosis.core.OsmosisRuntimeException: One or more tasks failed.
    at org.openstreetmap.osmosis.core.pipeline.common.Pipeline.waitForCompletion(Pipeline.java:146)
    at org.openstreetmap.osmosis.core.Osmosis.run(Osmosis.java:92)
    at org.openstreetmap.osmosis.core.Osmosis.main(Osmosis.java:37)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.base/java.lang.reflect.Method.invoke(Method.java:566)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launchStandard(Launcher.java:330)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:238)
    at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(Launcher.java:415)
    at org.codehaus.plexus.classworlds.launcher.Launcher.main(Launcher.java:356)
    at org.codehaus.classworlds.Launcher.main(Launcher.java:47)
isheff commented 3 years ago

I re-packed the data with gzip compression and now it seems to work. This would seem to confirm that the problem is in reading the bz2 compression.

HeytorNorberth commented 1 year ago

As you did?