Parsing planet-latest.osm.pbf fails

[Turbo87]

I downloaded http://ftp5.gwdg.de/pub/misc/openstreetmap/planet.openstreetmap.org/pbf/planet-latest.osm.pbf and tried to read it via:

            Osmium::OSMFile infile(argv[1]);
            osm::handler handler(argv[2]);
            infile.read(handler);

this threw an exception "Open failed". I checked the md5sum of the file and it was downloaded correctly. Any ideas what could cause this exception?! Smaller extracts are working fine (tested with north-rhine-westphalia.osm.pbf).

[joto]

The open() system call failed. Either the file is not there or you don't have the right permissions to read it.

[Turbo87]

Hmm... I would doubt that. The file is definitly there according to ls and nautilus and I don't think the permissions were wrong either. I downloaded it with wget from the same username. I did the same with the bz2 planet dump and processing that one worked instantly...

I'm running this on a 32bit machine, the file is 14GB... could that be the problem!?

[joto]

Ask the exception object for system_errno() that should tell you something. Or do an strace.

[Turbo87]

I stepped through it with gdb and I get the error code 75 in OSMFile::open_input_file().

from http://rabbit.eng.miami.edu/info/functions/errors.html:

define EPROGMISMATCH 75 /* Program version wrong */

Any idea what that should tell me?!

[joto]

That tells you that you are looking at the wrong place. On my Linux system /usr/include/asm-generic/errno.h says 75 is EOVERFLOW 75 Then man 2 open tells me: EOVERFLOW pathname refers to a regular file that is too large to be opened. The usual scenario here is that an application compiled on a 32-bit platform without -D_FILE_OFFSET_BITS=64 tried to open a file whose size exceeds (2<<31)-1 bits; see also O_LARGEFILE above. This is the error specified by POSIX.1-2001; in kernels before 2.6.24, Linux gave the error EFBIG for this case.

The example Makefiles provided use -D_FILE_OFFSET_BITS=64 .

[Turbo87]

Perfect! That did the trick. Thanks a lot! Should hopefully save me some time now compared to the bz2 parsing...