Project-OSRM / osrm-backend

Open Source Routing Machine - C++ backend
http://map.project-osrm.org
BSD 2-Clause "Simplified" License
6.42k stars 3.4k forks source link

Segmentation fault (core dumped) #1081

Closed escurzo closed 10 years ago

escurzo commented 10 years ago

Hi,

I have this error

oper@osrm:~/Project-OSRM$ ./osrm-datastore ../data/planet-latest.osrm
[info] load names from: "../data/planet-latest.osrm.names"
[info] size: 5190126
[info] allocating shared memory of 26923646936 bytes
[warn] could not delete DATA_1
[warn] could not delete LAYOUT_1
[info] all data loaded
oper@osrm:~/Project-OSRM$ ./osrm-routed --sharedmemory=yes
[info] starting up engines, v0.4.1, compiled at Jun 13 2014, 16:35:01
[info] loaded plugin: table
[info] loaded plugin: hello
[info] loaded plugin: locate
[info] loaded plugin: nearest
[info] loaded plugin: timestamp
[info] loaded plugin: viaroute
[info] http 1.1 compression handled by zlib version 1.2.8
[info] running and waiting for requests
[info] 13-06-2014 18:31:26 192.168.2.4 - Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/35.0.1916.153 Safari/537.36 /viaroute?loc=41.407366,2.217661&loc=41.398475,2.200613
Segmentation fault (core dumped)
oper@osrm:~/Project-OSRM$ ./osrm-routed --sharedmemory=yes
[info] starting up engines, v0.4.1, compiled at Jun 13 2014, 16:35:01
[info] loaded plugin: table
[info] loaded plugin: hello
[info] loaded plugin: locate
[info] loaded plugin: nearest
[info] loaded plugin: timestamp
[info] loaded plugin: viaroute
[info] http 1.1 compression handled by zlib version 1.2.8
[info] running and waiting for requests
[info] 13-06-2014 18:35:52 192.168.2.4 - Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/35.0.1916.153 Safari/537.36 /viaroute?z=18&output=json&jsonp=OSRM.JSONP.callbacks.route&checksum=-1246200715&loc=24.812646,46.503126&loc=24.737156,46.757469&instructions=false
[info] 13-06-2014 18:35:53 192.168.2.4 - Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/35.0.1916.153 Safari/537.36 /favicon.ico
[info] 13-06-2014 18:36:12 192.168.2.4 - Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/35.0.1916.153 Safari/537.36 /viaroute?z=18&output=json&loc=24.812646,46.503126&loc=24.737156,46.757469&instructions=false
[info] routing took 1ms
[info] descriptor took 0ms
[info] 13-06-2014 18:37:18 192.168.2.4 - Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/35.0.1916.153 Safari/537.36 /viaroute?z=18&output=json&loc=41.398475,2.200613&loc=41.480449,2.316999&instructions=false
[info] routing took 1ms
[info] descriptor took 0ms
[info] 13-06-2014 18:39:58 192.168.2.4 - Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/35.0.1916.153 Safari/537.36 /viaroute?z=14&output=json&loc=41.488070,2.067661&loc=41.475789,2.089376&instructions=false
Segmentation fault (core dumped)

Same url https://router.project-osrm.org/viaroute?z=14&output=json&loc=41.488070,2.067661&loc=41.475789,2.089376&instructions=false works fine.

Some machine parameters top - 18:44:27 up 21 min, 2 users, load average: 0.14, 0.09, 0.14 Tasks: 100 total, 1 running, 99 sleeping, 0 stopped, 0 zombie %Cpu(s): 0.0 us, 0.0 sy, 0.0 ni,100.0 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 st KiB Mem: 29355984 total, 29198168 used, 157816 free, 4856 buffers KiB Swap: 16774140 total, 0 used, 16774140 free. 28801940 cached Mem

I use ubuntu 14.04

Thanks

alex85k commented 10 years ago

Are you using data files and osrm-datastore from the same release as osrm-routed version?

If the errors are repeating, you could try to build debug version (cmake .. -DCMake_BUILD_TYPE=Debug; make -j2) and run it via gdb:

gdb ./osrm-routed
run --sharedmemory=yes

When it breaks with segfault, the details will be printed (if no, send backtrace command to gdb).

escurzo commented 10 years ago

Yes diferent machine but same osrm version

El viernes, 13 de junio de 2014, alex85k notifications@github.com escribió:

Are you data files from the same release as osrm-routed version?

If the errors are repeating, you could try to build debug version (cmake .. -DCMake_BUILD_TYPE=Debug) and run it via gdb:

gdb ./osrm-routed run --sharedmemory=yes

When it breaks with segfault, the details will be printed (if no, send backtrace command to gdb).

— Reply to this email directly or view it on GitHub https://github.com/DennisOSRM/Project-OSRM/issues/1081#issuecomment-46036058 .

Josep Ferrer CTO

IGGEOS EU: +34 93 176 02 15 / +34 91 119 63 01 US: +1 (415) 376 4112 www.iggeos.com

DennisOSRM commented 10 years ago

@escurzo do I get this right, that some queries work fine and others fail?

escurzo commented 10 years ago

@DennisOSRM all queries have invalid results but only some queries prodices Segmentation fault

This is the backtrace

oper@osrm:~/Project-OSRM$ ./osrm-routed --sharedmemory=yes
[info] starting up engines, v0.4.1, compiled at Jun 13 2014, 16:35:01
[warn] exception: no leaf index file given. Is any data loaded into shared memory?
oper@osrm:~/Project-OSRM$ ./osrm-
osrm-datastore  osrm-extract    osrm-prepare    osrm-routed     
oper@osrm:~/Project-OSRM$ ./osrm-datastore ../data/planet-latest.osrm
[info] load names from: "../data/planet-latest.osrm.names"
[info] size: 5190126
[info] allocating shared memory of 26923646936 bytes
[warn] could not delete DATA_1
[warn] could not delete LAYOUT_1
[info] all data loaded
oper@osrm:~/Project-OSRM$ gdb ./osrm-routed
GNU gdb (Ubuntu 7.7-0ubuntu3.1) 7.7
Copyright (C) 2014 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
<http://www.gnu.org/software/gdb/documentation/>.
For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from ./osrm-routed...(no debugging symbols found)...done.
(gdb) run --sharedmemory=yes
Starting program: /home/oper/Project-OSRM/osrm-routed --sharedmemory=yes
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
[info] starting up engines, v0.4.1, compiled at Jun 13 2014, 16:35:01
[info] loaded plugin: table
[info] loaded plugin: hello
[info] loaded plugin: locate
[info] loaded plugin: nearest
[info] loaded plugin: timestamp
[info] loaded plugin: viaroute
[info] http 1.1 compression handled by zlib version 1.2.8
[New Thread 0x7ff9ae8b8700 (LWP 1092)]
[info] running and waiting for requests
[New Thread 0x7ff9ae0b7700 (LWP 1093)]
[New Thread 0x7ff9ad8b6700 (LWP 1094)]
[New Thread 0x7ff9ad0b5700 (LWP 1095)]
[New Thread 0x7ff9ac8b4700 (LWP 1096)]
[New Thread 0x7ff9a7fff700 (LWP 1097)]
[info] 16-06-2014 15:14:15 192.168.2.4 - Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/35.0.1916.153 Safari/537.36 /viaroute?z=14&output=json&loc=41.488070,2.067661&loc=41.475789,2.089376&instructions=false

Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0x7ff9ae0b7700 (LWP 1093)]
0x000000000045f5a8 in BasicRoutingInterface<BaseDataFacade<QueryEdge::EdgeData> >::UnpackEdge(unsigned int, unsigned int, std::vector<unsigned int, std::allocator<unsigned int> >&) const ()
(gdb) backtrace
#0  0x000000000045f5a8 in BasicRoutingInterface<BaseDataFacade<QueryEdge::EdgeData> >::UnpackEdge(unsigned int, unsigned int, std::vector<unsigned int, std::allocator<unsigned int> >&) const ()
#1  0x000000000046b62a in AlternativeRouting<BaseDataFacade<QueryEdge::EdgeData> >::operator()(PhantomNodes const&, RawRouteData&) ()
#2  0x00000000004794c6 in ViaRoutePlugin<BaseDataFacade<QueryEdge::EdgeData> >::HandleRequest(RouteParameters const&, http::Reply&) ()
#3  0x000000000047b0d0 in OSRM_impl::RunQuery(RouteParameters&, http::Reply&) ()
#4  0x000000000047c443 in RequestHandler::handle_request(http::Request const&, http::Reply&) ()
#5  0x000000000047d354 in http::Connection::handle_read(boost::system::error_code const&, unsigned long) ()
#6  0x000000000048ae97 in boost::asio::detail::completion_handler<boost::asio::detail::rewrapped_handler<boost::asio::detail::binder2<boost::asio::detail::wrapped_handler<boost::asio::io_service::strand, boost::_bi::bind_t<void, boost::_mfi::mf2<void, http::Connection, boost::system::error_code const&, unsigned long>, boost::_bi::list3<boost::_bi::value<std::shared_ptr<http::Connection> >, boost::arg<1> (*)(), boost::arg<2> (*)()> >, boost::asio::detail::is_continuation_if_running>, boost::system::error_code, unsigned long>, boost::_bi::bind_t<void, boost::_mfi::mf2<void, http::Connection, boost::system::error_code const&, unsigned long>, boost::_bi::list3<boost::_bi::value<std::shared_ptr<http::Connection> >, boost::arg<1> (*)(), boost::arg<2> (*)()> > > >::do_complete(boost::asio::detail::task_io_service*, boost::asio::detail::task_io_service_operation*, boost::system::error_code const&, unsigned long) ()
#7  0x000000000048bbc8 in boost::asio::detail::reactive_socket_recv_op<boost::asio::mutable_buffers_1, boost::asio::detail::wrapped_handler<boost::asio::io_service::strand, boost::_bi::bind_t<void, boost::_mfi::mf2<void, http::Connection, boost::system::error_code const&, unsigned long>, boost::_bi::list3<boost::_bi::value<std::shared_ptr<http::Connection> >, boost::arg<1> (*)(), boost::arg<2> (*)()> >, boost::asio::detail::is_continuation_if_running> >::do_complete(boost::asio::detail::task_io_service*, boost::asio::detail::task_io_service_operation*, boost::system::error_code const&, unsigned long) ()
#8  0x000000000048cf34 in boost::asio::detail::epoll_reactor::descriptor_state::do_complete(boost::asio::detail::task_io_service*, boost::asio::detail::task_io_service_operation*, boost::system::error_code const&, unsigned long) ()
#9  0x000000000048c6e7 in boost::asio::io_service::run() ()
#10 0x00007ffff63c4bf0 in ?? () from /usr/lib/x86_64-linux-gnu/libstdc++.so.6
#11 0x00007ffff6a40182 in start_thread (arg=0x7ff9ae0b7700) at pthread_create.c:312
#12 0x00007ffff7b0f30d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111
(gdb) 
DennisOSRM commented 10 years ago

looks like the data is broken.

escurzo commented 10 years ago

Looking at the log it seems that the generation of data worked well!

Someone I can pass the planet data in OSRM format for to do some tests please? My OSRM version is v0.4.1, compiled at Jun 13 2014, 16:35:01

[info] generating edge-expanded edges . 10% . 20% . 30% . 40% . 50% . 60% . 70% . 80% . 90% . 100% [info] Generated 506417744 edge based nodes [info] Node-based graph contains 209269532 edges [info] Edge-expanded graph ... [info] contains 421095960 edges [info] skips 141460 turns, defined by 260742 restrictions [info] skips 179319580 U turns [info] skips 203348 turns over barriers [info] Timing statistics for edge-expanded graph: [info] Geometry compression: 698.107s [info] Renumbering edges: 2.65s [info] Generating nodes: 233.761s [info] Generating edges: 428.706s [info] building r-tree ... [info] constructing r-tree of 506417744 edge elements build on-top of 479684747 coordinates [info] finished r-tree construction in 848.53 seconds [info] using hardware based CRC32 computation [info] CRC32: 354183759 [info] writing node map ... [info] initializing contractor merged 637746 edges out of 842191920 contractor finished initalization Contractor is using 8 threads initializing elimination PQ ...ok preprocessing 209269532 nodes .... 10% . 20% . 30% . 40% . 50% . 60% . [flush 136928945 nodes] 70% . 80% . 90% . 100% [info] Contraction took 13012.3 sec [info] Getting edges of minimized graph . 10% . 20% . 30% . 40% . 50% . 60% . 70% . 80% . 90% . 100% [info] Serializing compacted graph of 1170001723 edges [info] Building node array [info] Building edge array [info] Preprocessing : 16454.9 seconds [info] Expansion : 219072 nodes/sec and 95573.5 edges/sec [info] Contraction: 16082.4 nodes/sec and 89915.1 edges/sec [info] finished preprocessing

DennisOSRM commented 10 years ago

What are the file sizes of your data files.

escurzo commented 10 years ago

-rw-rw-r-- 1 oper oper 22368465228 Jun 12 19:26 planet-latest.osrm -rw-rw-r-- 1 oper oper 5053151524 Jun 12 20:01 planet-latest.osrm.edges -rw-rw-r-- 1 oper oper 22284377948 Jun 12 20:17 planet-latest.osrm.fileIndex -rw-rw-r-- 1 oper oper 4122249420 Jun 12 20:02 planet-latest.osrm.geometry -rw-rw-r-- 1 oper oper 14877098976 Jun 13 00:01 planet-latest.osrm.hsgr -rw-rw-r-- 1 oper oper 105656405 Jun 12 19:26 planet-latest.osrm.names -rw-rw-r-- 1 oper oper 5756216968 Jun 12 20:18 planet-latest.osrm.nodes -rw-rw-r-- 1 oper oper 138662680 Jun 12 20:17 planet-latest.osrm.ramIndex -rw-rw-r-- 1 oper oper 4185920 Jun 12 13:34 planet-latest.osrm.restrictions

Are correct?

alex85k commented 10 years ago

Did you try to run osrm-routed directly from file, not with datastore?

escurzo commented 10 years ago

oper@osrm:~/Project-OSRM$ ./osrm-routed ../data/planet-latest.osrm [info] starting up engines, v0.4.1, compiled at Jun 13 2014, 16:35:01 [info] HSGR file: "../data/planet-latest.osrm.hsgr" [info] loading graph data [info] loading graph from ../data/planet-latest.osrm.hsgr [info] number_of_nodes: 209269533, number_of_edges: 1170001723 [info] loaded 209269534 nodes and 1170001723 edges [info] Data checksum is 354183759 [info] loading egde information [info] loading geometries [info] loading r-tree [info] loading timestamp [info] loading street names [info] loaded plugin: table [info] loaded plugin: hello [info] loaded plugin: locate [info] loaded plugin: nearest [info] loaded plugin: timestamp [info] loaded plugin: viaroute [info] http 1.1 compression handled by zlib version 1.2.8 [info] running and waiting for requests

[info] 17-06-2014 10:45:25 192.168.2.4 - Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/35.0.1916.153 Safari/537.36 /viaroute?z=14&output=json&loc=41.488070,2.067661&loc=41.475789,2.089376&instructions=false Segmentation fault (core dumped)

In my /var/log/syslog Jun 17 10:35:26 osrm kernel: [ 3.277827] IPv6: ADDRCONF(NETDEV_UP): eth2: link is not ready Jun 17 10:35:26 osrm kernel: [ 3.277831] IPv6: ADDRCONF(NETDEV_UP): eth3: link is not ready Jun 17 10:35:26 osrm kernel: [ 3.340855] EXT4-fs (xvda1): re-mounted. Opts: errors=remount-ro Jun 17 10:35:26 osrm kernel: [ 3.472886] lp: driver loaded but no devices found Jun 17 10:35:26 osrm kernel: [ 3.570777] type=1400 audit(1402994125.810:2): apparmor="STATUS" operation="profile_load" profile="unconfined" name="/sbin/dhclient" pid=384 comm="apparmor_parser" Jun 17 10:35:26 osrm kernel: [ 3.570785] type=1400 audit(1402994125.810:3): apparmor="STATUS" operation="profile_load" profile="unconfined" name="/usr/lib/NetworkManager/nm-dhcp-client.action" pid=384 comm="apparmor_parser" Jun 17 10:35:26 osrm kernel: [ 3.570790] type=1400 audit(1402994125.810:4): apparmor="STATUS" operation="profile_load" profile="unconfined" name="/usr/lib/connman/scripts/dhclient-script" pid=384 comm="apparmor_parser" Jun 17 10:35:26 osrm kernel: [ 3.571323] type=1400 audit(1402994125.810:5): apparmor="STATUS" operation="profile_replace" profile="unconfined" name="/usr/lib/NetworkManager/nm-dhcp-client.action" pid=384 comm="apparmor_parser" Jun 17 10:35:26 osrm kernel: [ 3.571330] type=1400 audit(1402994125.810:6): apparmor="STATUS" operation="profile_replace" profile="unconfined" name="/usr/lib/connman/scripts/dhclient-script" pid=384 comm="apparmor_parser" Jun 17 10:35:26 osrm kernel: [ 3.571586] type=1400 audit(1402994125.810:7): apparmor="STATUS" operation="profile_replace" profile="unconfined" name="/usr/lib/connman/scripts/dhclient-script" pid=384 comm="apparmor_parser" Jun 17 10:35:26 osrm kernel: [ 3.574628] type=1400 audit(1402994125.814:8): apparmor="STATUS" operation="profile_replace" profile="unconfined" name="/sbin/dhclient" pid=386 comm="apparmor_parser" Jun 17 10:35:26 osrm kernel: [ 3.574637] type=1400 audit(1402994125.814:9): apparmor="STATUS" operation="profile_replace" profile="unconfined" name="/usr/lib/NetworkManager/nm-dhcp-client.action" pid=386 comm="apparmor_parser" Jun 17 10:35:26 osrm kernel: [ 3.574642] type=1400 audit(1402994125.814:10): apparmor="STATUS" operation="profile_replace" profile="unconfined" name="/usr/lib/connman/scripts/dhclient-script" pid=386 comm="apparmor_parser" Jun 17 10:35:26 osrm rsyslogd-2039: Could no open output pipe '/dev/xconsole': No such file or directory [try http://www.rsyslog.com/e/2039 ] Jun 17 10:35:26 osrm acpid: cannot open input layer Jun 17 10:35:26 osrm cron[748]: (CRON) INFO (pidfile fd = 3) Jun 17 10:35:26 osrm cron[815]: (CRON) STARTUP (fork ok) Jun 17 10:35:26 osrm acpid: starting up with netlink and the input layer Jun 17 10:35:26 osrm cron[815]: (CRON) INFO (Running @reboot jobs) Jun 17 10:35:26 osrm acpid: 1 rule loaded Jun 17 10:35:26 osrm acpid: waiting for events: event logging is off Jun 17 10:35:32 osrm ntpdate[603]: step time server 91.189.94.4 offset 0.019535 sec Jun 17 10:35:48 osrm ntpdate[986]: adjust time server 91.189.94.4 offset 0.000734 sec Jun 17 10:37:21 osrm kernel: [ 119.645321] Adding 314571772k swap on /dev/xvdb1. Priority:-2 extents:1 across:314571772k SSFS Jun 17 10:45:26 osrm kernel: [ 603.808610] show_signal_msg: 48 callbacks suppressed Jun 17 10:45:26 osrm kernel: [ 603.808617] osrm-routed[1072]: segfault at 7f3f196d400b ip 000000000045f5a8 sp 00007f3049e4d020 error 4 in osrm-routed[400000+af000]

DennisOSRM commented 10 years ago

File sizes look ok, but running and preprocessing on different machines may have issues such as you are experiencing. Especially if you use a 32bit architecture somewhere.

escurzo commented 10 years ago

I was tested OSRM in the same machine that I have processes files and the process crash

ubuntu@ip-172-31-27-7:~/Project-OSRM$ ./osrm-routed /data/planet-latest.osrm [warn] ./osrm-routed could not be locked to RAM [info] starting up engines, v0.4.1, compiled at Jun 17 2014, 10:16:10 [info] HSGR file: "/data/planet-latest.osrm.hsgr" [info] loading graph data [info] loading graph from /data/planet-latest.osrm.hsgr [info] number_of_nodes: 209269533, number_of_edges: 1170001723 T[info] loaded 209269534 nodes and 1170001723 edges [info] Data checksum is 354183759 [info] loading egde information [info] loading geometries [info] loading r-tree [info] loading timestamp [info] loading street names [info] loaded plugin: table [info] loaded plugin: hello [info] loaded plugin: locate [info] loaded plugin: nearest [info] loaded plugin: timestamp [info] loaded plugin: viaroute [info] http 1.1 compression handled by zlib version 1.2.8 [info] running and waiting for requests [info] 17-06-2014 10:38:22 80.28.209.138 - Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/35.0.1916.153 Safari/537.36 /viaroute?z=14&output=json&loc=41.488070,2.067661&loc=41.475789,2.089376&instructions=false Segmentation fault (core dumped)

the system log shows Jun 17 10:11:53 ip-172-31-27-7 kernel: [ 7.431832] ISOFS: Unable to identify CD-ROM format. Jun 17 10:11:53 ip-172-31-27-7 rsyslogd-2039: Could no open output pipe '/dev/xconsole': No such file or directory [try http://www.rsyslog.com/e/2039 ] Jun 17 10:11:53 ip-172-31-27-7 dbus[1193]: [system] AppArmor D-Bus mediation is enabled Jun 17 10:11:53 ip-172-31-27-7 kernel: [ 8.100084] audit_printk_skb: 9 callbacks suppressed Jun 17 10:11:53 ip-172-31-27-7 kernel: [ 8.100088] type=1400 audit(1402999913.825:14): apparmor="STATUS" operation="profile_replace" profile="unconfined" name="/sbin/dhclient" pid=1239 comm="apparmor_parser" Jun 17 10:11:53 ip-172-31-27-7 kernel: [ 8.100096] type=1400 audit(1402999913.825:15): apparmor="STATUS" operation="profile_replace" profile="unconfined" name="/usr/lib/NetworkManager/nm-dhcp-client.action" pid=1239 comm="apparmor_parser" Jun 17 10:11:53 ip-172-31-27-7 kernel: [ 8.100102] type=1400 audit(1402999913.825:16): apparmor="STATUS" operation="profile_replace" profile="unconfined" name="/usr/lib/connman/scripts/dhclient-script" pid=1239 comm="apparmor_parser" Jun 17 10:11:53 ip-172-31-27-7 kernel: [ 8.100569] type=1400 audit(1402999913.825:17): apparmor="STATUS" operation="profile_replace" profile="unconfined" name="/usr/lib/NetworkManager/nm-dhcp-client.action" pid=1239 comm="apparmor_parser" Jun 17 10:11:53 ip-172-31-27-7 kernel: [ 8.100575] type=1400 audit(1402999913.825:18): apparmor="STATUS" operation="profile_replace" profile="unconfined" name="/usr/lib/connman/scripts/dhclient-script" pid=1239 comm="apparmor_parser" Jun 17 10:11:53 ip-172-31-27-7 kernel: [ 8.100812] type=1400 audit(1402999913.825:19): apparmor="STATUS" operation="profile_replace" profile="unconfined" name="/usr/lib/connman/scripts/dhclient-script" pid=1239 comm="apparmor_parser" Jun 17 10:11:53 ip-172-31-27-7 kernel: [ 8.105861] type=1400 audit(1402999913.829:20): apparmor="STATUS" operation="profile_load" profile="unconfined" name="/usr/sbin/tcpdump" pid=1241 comm="apparmor_parser" Jun 17 10:11:54 ip-172-31-27-7 pollinate[1306]: system was previously seeded at [2014-06-11 14:34:42.806643000 +0000] Jun 17 10:11:54 ip-172-31-27-7 pollinate[1309]: To re-seed this system again, use the -r|--reseed option Jun 17 10:11:54 ip-172-31-27-7 pollinate[1378]: system was previously seeded at [2014-06-11 14:34:42.806643000 +0000] Jun 17 10:11:54 ip-172-31-27-7 acpid: starting up with netlink and the input layer Jun 17 10:11:54 ip-172-31-27-7 pollinate[1381]: To re-seed this system again, use the -r|--reseed option Jun 17 10:11:54 ip-172-31-27-7 cron[1271]: (CRON) INFO (pidfile fd = 3) Jun 17 10:11:54 ip-172-31-27-7 acpid: 1 rule loaded Jun 17 10:11:54 ip-172-31-27-7 acpid: waiting for events: event logging is off Jun 17 10:11:54 ip-172-31-27-7 cron[1389]: (CRON) STARTUP (fork ok) Jun 17 10:11:54 ip-172-31-27-7 cron[1389]: (CRON) INFO (Running @reboot jobs) Jun 17 10:11:59 ip-172-31-27-7 ntpdate[963]: step time server 91.189.94.4 offset 0.874286 sec Jun 17 10:12:06 ip-172-31-27-7 ntpdate[1499]: adjust time server 91.189.94.4 offset -0.000206 sec Jun 17 10:12:58 ip-172-31-27-7 kernel: [ 71.871860] EXT4-fs (xvdf): mounted filesystem with ordered data mode. Opts: (null) Jun 17 10:17:01 ip-172-31-27-7 CRON[2391]: (root) CMD ( cd / && run-parts --report /etc/cron.hourly) Jun 17 10:38:09 ip-172-31-27-7 dhclient: DHCPREQUEST of 172.31.27.7 on eth0 to 172.31.16.1 port 67 (xid=0x69de4aec) Jun 17 10:38:09 ip-172-31-27-7 dhclient: DHCPACK of 172.31.27.7 from 172.31.16.1 Jun 17 10:38:09 ip-172-31-27-7 dhclient: bound to 172.31.27.7 -- renewal in 1710 seconds. Jun 17 10:38:22 ip-172-31-27-7 kernel: [ 1595.811003] osrm-routed[2607]: segfault at 7f3af4ae100b ip 00000000004616c8 sp 00007f2c1e7f9fb0 error 4 in osrm-routed[400000+af000]

There is an amazon i2.4xlarge instance with ubuntu 14.04

escurzo commented 10 years ago

Revising logs during procesing files I can see [STXXL-ERRMSG] External memory block allocation error: 2097152 bytes requested, 0 bytes free. Trying to extend the external memory space...

DennisOSRM commented 10 years ago

rebuild with debug configuration. osrm-routed will then inspect the data files very thoroughly and report if they are broken (which I suspect). If they are broken then you need to check configuration of swap, /tmp as well as the stxxl files.

emiltin commented 10 years ago

inspection of input would be a nice option in the release version, ie:

osrm-routed planet.osrm --inspect
DennisOSRM commented 10 years ago

Perhaps we could have this as a separate tool.

emiltin commented 10 years ago

would probably make sense

DennisOSRM commented 10 years ago

This is now in develop branch.