Error starting from osm.pbf file

[idanmid15]

Hi, recently I've seen the file which I used to init from is no longer found at its bz2 format. I tried running from the pbf file after reading the README which shows how to convert using osmium with the command:

docker run \             
    -e OVERPASS_META=yes \
    -e OVERPASS_MODE=init \
    -e OVERPASS_PLANET_URL=http://download.geofabrik.de/asia/israel-and-palestine-latest.osm.pbf \
    -e OVERPASS_DIFF_URL=http://download.openstreetmap.fr/replication/asia/israel/minute/ \
    -e OVERPASS_RULES_LOAD=10 \
    -e OVERPASS_COMPRESSION=gz \
    -e OVERPASS_UPDATE_SLEEP=3600 \
    -e OVERPASS_PLANET_PREPROCESS='mv /db/planet.osm.bz2 /db/planet.osm.pbf && osmium cat /db/planet.osm.pbf -o /db/planet.osm.bz2 && rm /db/planet.osm.pbf' \
    -v /big/docker/overpass_db/:/db \
    -p 12347:80 \
    -i -t \
    --name overpass_israel wiktorn/overpass-api

but I'm getting Running preprocessing command: mv /db/planet.osm.bz2 /db/planet.osm.pbf && osmium cat /db/planet.osm.pbf -o /db/planet.osm.bz2 && rm /db/planet.osm.pbf [======================================================================] 100% Reading XML file ... finished reading nodes. File error caught: /db/db/nodes.bin.idx File_Blocks_Index: Data file size does not match block size Failed to process planet file

Any idea why?

[wiktorn]

Which version of the docker image / Overpass-API are you running? If it's not 0.7.58.3 then please update to this version and try again.

[idanmid15]

Just tried with version 0.7.58.3 which unfortunately results in the same output

[idanmid15]

The bz2 file returned to the server and it's now happening with that file: docker run \ -e OVERPASS_META=yes \ -e OVERPASS_MODE=init \ -e OVERPASS_PLANET_URL=http://download.geofabrik.de/asia/israel-and-palestine-latest.osm.bz2 \ -e OVERPASS_DIFF_URL=http://download.openstreetmap.fr/replication/asia/israel/minute/ \ -e OVERPASS_RULES_LOAD=10 \ -v /big/docker/overpass_db/:/db \ -p 12345:80 \ -i -t \ --name overpass_api wiktorn/overpass-api:latest

Results in: Reading XML file ... finished reading nodes. File error caught: /db/db/nodes.bin.idx File_Blocks_Index: Data file size does not match block size

[wiktorn]

Just tried, the process is not yet finished:

 docker run  -e OVERPASS_META=yes  -e OVERPASS_MODE=init  -e OVERPASS_PLANET_URL=http://download.geofabrik.de/asia/israel-and-palestine-latest.osm.bz2  -e OVERPASS_DIFF_URL=http://download.openstreetmap.fr/replication/asia/israel/minute/  -e OVERPASS_RULES_LOAD=10  -v /big/docker/overpass_db99/:/db  -p 12347:80  -i -t  --name overpass_99 wiktorn/overpass-api:0.7.58.3
No database directory. Initializing
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  147M  100  147M    0     0  18.5M      0  0:00:07  0:00:07 --:--:-- 17.7M
Reading XML file ... finished reading nodes. Compute current ... ready. Flushing to database ....... done.
Reading XML file ... finished reading ways. Compute current ... ready. Flushing to database ....... done.
Reading XML file ... finished reading relations.
[...]
Flushing to database ....... done.
Update complete.
Database created. Now updating it.

So it looks like it works for me. I don't know what might be the issue on your end. The only difference is that I used explicit tag.

[idanmid15]

Found the problem! Thanks for confirming the command works correctly on your end. The issue was solved as soon as I changed the volume path to a new one. /big/docker/overpass_api must have had leftovers from the previous image, so starting out completely clean here solved it. Thanks!

[wiktorn]

Thanks for reporting!