search

bmander / graphserver

An open source multimodal trip planning engine
http://bmander.github.com/graphserver
Other
122 stars 60 forks source link

gs_compile_gdb hangs on 64-bit machine #13

Closed msoulier closed 14 years ago

msoulier commented 14 years ago

On a 64-bit platform I'm finding that I am unable to build a graphdb, as it is hanging part way through, chewing 100% cpu and never finishing.

It's been suggested that this is an sqlite3 on 64-bit problem, but I cannot find an existing report upstream.

msoulier commented 14 years ago

Looks like python is spinning its wheels reading and seeking.

read(3, "\r\0\0\0\27\0:\0\3\326\3\254\3\202\3X\3.\3\4\2\332\2\260\2\206\2\0022\2\10\1"..., 1024) = 1024 lseek(3, 4426752, SEEK_SET) = 4426752 read(3, "\5\0\0\0b\1R\0\0\0\21\34\1R\1Y\1`\1g\1n\1u\1|\1\203\1\212\1\221\1"..., 1024) = 1024 lseek(3, 4382720, SEEK_SET) = 4382720 read(3, "\r\0\0\0\32\0R\0\3\326\3\254\3\202\3X\3.\3\4\2\351\2\316\2\244\2z\2P\2&\1"..., 1024) = 1024 lseek(3, 4383744, SEEK_SET) = 4383744 read(3, "\r\0\0\0\27\0X\0\3\345\3\273\3\221\3g\3=\3\23\2\351\2\277\2\225\2k\2A\2\27\1"..., 1024) = 1024 lseek(3, 4384768, SEEK_SET) = 4384768 read(3, "\r\0\0\0\27\0I\0\3\326\3\254\3\202\3X\3.\3\4\2\332\2\260\2\206\2\0022\2\10\1"..., 1024) = 1024 lseek(3, 4385792, SEEK_SET) = 4385792 read(3, "\r\0\0\0\27\0X\0\3\326\3\254\3\202\3X\3.\3\4\2\332\2\260\2\206\2\0022\2\10\1"..., 1024) = 1024 lseek(3, 4386816, SEEK_SET) = 4386816 read(3, "\r\0\0\0\27\0I\0\3\326\3\254\3\202\3X\3.\3\4\2\332\2\260\2\206\2\2A\2\27\1"..., 1024) = 1024 lseek(3, 4387840, SEEK_SET) = 4387840 read(3, "\r\0\0\0\32\0a\0\3\326\3\254\3\202\3X\3.\3\23\2\351\2\277\2\225\2k\2A\2\27\1"..., 1024) = 1024 lseek(3, 4388864, SEEK_SET) = 4388864

Would this be reading from the gtfsdb?

Ah yes it is.

So, would this be some kind of sqlite3 problem?

msoulier commented 14 years ago

BTW it always seems to happen while loading the trip bundles.

Loading trip bundles into graph... 0/11786 trip bundles loaded 118/11786 trip bundles loaded 236/11786 trip bundles loaded 354/11786 trip bundles loaded 472/11786 trip bundles loaded 590/11786 trip bundles loaded

etc etc

7670/11786 trip bundles loaded 7788/11786 trip bundles loaded

and then it hangs. 100% cpu and 30% memory but it goes no further.

bmander commented 14 years ago

gs_compile_gdb is depricated - it isn't even in the codebase. Behold: http://github.com/bmander/graphserver/tree/master/pygs/graphserver/compiler/. If you're using it you're likely using an old binary still installed on your system, but abandoned by the codebase.

msoulier commented 14 years ago

I doubt that's relevant, I think it's an sqlite issue. Since you heavily depend on sqlite I thought you'd like to track the issue and where it's fixed.

msoulier commented 14 years ago

Is there a reason why the new compile_graph.py code isn't installed as a tool in bin when you install graphserver? I would think that compiling a GTFS zip to a db and then to a graph db would be a common thing.

Thanks.

msoulier commented 14 years ago

Whoops, shouldn't be updating this.

This issue turned out to be a 64 bit issue with sqlite. I picked up the latest sqlite and pysqlite and it went away.