Open tsworman opened 6 years ago
daniel-j-h
commented
6 years ago As discussed in IRC this looks like a Windows + Docker-Toolkit problem.
In the meantime try using a Linux box or use the native Windows binaries.
As for next actions here: someone with a Windows box needs to debug this (cc @oxidase ?).
oxidase
commented
6 years ago @tsworman @daniel-j-h the issue is caused by a mmap fail in osrm-partition with PROT_WRITE and MAP_SHARED flags.
open("/data/Downloads/monaco-latest.osrm.fileIndex", O_RDWR) = 3
fstat(3, {st_mode=S_IFREG|0777, st_size=97520, ...}) = 0
mmap(NULL, 97520, PROT_READ|PROT_WRITE, MAP_SHARED, 3, 0) = -1 EINVAL (Invalid argument)At the same time osrm-routed without PROT_WRITE flag works fine
open("/data/Downloads/monaco-latest.osrm.fileIndex", O_RDONLY) = 3
fstat(3, {st_mode=S_IFREG|0777, st_size=97520, ...}) = 0
mmap(NULL, 97520, PROT_READ, MAP_SHARED, 3, 0) = 0x7f65a2d27000Similar issue https://stackoverflow.com/a/18421071 and potentially related issue in VirtualBox https://www.virtualbox.org/ticket/819
A workaround fix can be a read-modify-write renumbering instead of the current inplace one /cc @TheMarex
ibrt
commented
5 years ago Interesting, I ran into this issue using a Ubuntu guest in VirtualBox on macOS host. I was running the Docker OSRM container in the guest OS. I figured it happens when the fileIndex is inside a VirtualBox shared directory. If I put the fileIndex somewhere else, it works fine. Likely some issue with vboxfs.
grib0
commented
5 years ago Probably linked to #5086 a chmod 644 fileIndex should work
alitarraf
commented
5 years ago EDIT: I uninstalled docker toolbox, and virtual box. Then upgraded to WIN 10 PRO. Then installed Docker for Windows. Everything works like a charm now with Hyper-V. I think the error is related to virtual box/windows. Dont have any errors now.
I am having a similar issue. Running Win 10 Home, with Docker Toolbox. However, I get an mmap_file.hpp:60 failure after the R-tree construction in the osrm-extract step.
I have tried latest docker image and v5.21.0 and v5.20.0 with similar result.
Here is my console output.
PS D:\Documents\Projects\OSRM-backend\try-docker\data> docker run -t -v //d/Documents/Projects/OSRM-backen d/try-docker/data/:/data osrm/osrm-backend:v5.20.0 osrm-extract -p /opt/car.lua /data/monaco-latest.osm.pb f [info] Parsed 0 location-dependent features with 0 GeoJSON polygons [info] Using script /opt/car.lua [info] Input file: monaco-latest.osm.pbf [info] Profile: car.lua [info] Threads: 1 [info] Parsing in progress.. [info] input file generated by osmium/1.8.0 [info] timestamp: 2019-03-01T21:15:02Z [info] Using profile api version 4 [info] Found 3 turn restriction tags: [info] motorcar [info] motor_vehicle [info] vehicle [info] Parse relations ... [info] Parse ways and nodes ... [info] Parsing finished after 0.328125 seconds [info] Raw input contains 43726 nodes, 3175 ways, and 26 relations, 44 restrictions [info] Sorting used nodes ... ok, after 0.000231s [info] Erasing duplicate nodes ... ok, after 1.5e-05s [info] Sorting all nodes ... ok, after 5.9e-05s [info] Building node id map ... ok, after 8.3e-05s [info] Confirming/Writing used nodes ... ok, after 0.00833s [info] Writing barrier nodes ... ok, after 0s [info] Writing traffic light nodes ... ok, after 0s [info] Processed 4836 nodes [info] Sorting edges by start ... ok, after 0.000303s [info] Setting start coords ... ok, after 0.000297s [info] Sorting edges by target ... ok, after 0.000257s [info] Computing edge weights ... ok, after 0.001121s [info] Sorting edges by renumbered start ... ok, after 0.000344s [info] Writing used edges ... ok, after 0.000918s -- Processed 5056 edges [info] Writing way meta-data ... ok, after 0.000297s -- Metadata contains << 837 entries. [info] Sorting used ways ... ok, after 7.1e-05s [info] Collecting start/end information on 0 maneuver overrides...ok, after 8e-06s [info] Collecting start/end information on 0 maneuver overrides...ok, after 2e-06s [info] Collecting start/end information on 44 restrictions...ok, after 3.5e-05s [info] Collecting start/end information on 44 restrictions...ok, after 3e-05s [info] writing street name index ... ok, after 0.001513s [info] extraction finished after 0.366118s [info] Generating edge-expanded graph representation [info] . 10% . 20% . 30% . 40% . 50% . 60% . 70% . 80% . 90% . 100% [info] Node compression ratio: 0.167494 [info] Edge compression ratio: 0.20214 [info] graph compression removed 629 annotations of 837 in 0.000332 seconds [info] Find segregated edges in node-based graph ... [info] ok, after 0.003219s [info] Segregated edges count = 27 [info] Writing nodes for nodes-based and edges-based graphs ... [info] Geometry successfully removed: compressed edges: 2100 compressed geometries: 10102 longest chain length: 46 cmpr ratio: 0.20788 avg chain length: 4.81048 [info] Generating edge expanded nodes ... [info] . 10% . 20% . 30% . 40% . 50% . 60% . 70% . 80% . 90% . 100% [info] Expanding via-way turn restrictions ... [info] [info] Generated 1445 nodes (0 of which are duplicates) and 5046 segments in edge-expanded graph [info] Generating edge-expanded edges [info] . 10% . 20% . 30% . 40% . 50% . 60% . 70% . 80% . 90% . 100% [info] Sorting and writing 0 maneuver overrides... [info] done. [info] Renumbering turns [info] Writing 0 conditional turn penalties... [info] Generated 5046 edge based node segments [info] Node-based graph contains 1445 edges [info] Edge-expanded graph ... [info] contains 2131 edges [info] Timing statistics for edge-expanded graph: [info] Renumbering edges: 4.9e-05s [info] Generating nodes: 0.007678s [info] Generating edges: 0.058718s [info] Generating guidance turns [info] . 10% . 20% . 30% . 40% . 50% . 60% . 70% . 80% . 90% . 100% [info] done. [info] Created 15 entry classes and 476 Bearing Classes [info] Handled: 7 of 31 lanes: 22.5806 %. [info] Assigned 2631 turn instruction types: [info] new name: 179 (6.80%) [info] continue: 201 (7.64%) [info] turn: 766 (29.11%) [info] fork: 48 (1.82%) [info] end of road: 223 (8.48%) [info] enter roundabout: 62 (2.36%) [info] enter and exit roundabout: 6 (0.23%) [info] enter rotary: 3 (0.11%) [info] enter and exit rotary: 2 (0.08%) [info] enter roundabout turn: 15 (0.57%) [info] (noturn): 301 (11.44%) [info] (suppressed): 631 (23.98%) [info] roundabout: 6 (0.23%) [info] exit roundabout: 76 (2.89%) [info] rotary: 2 (0.08%) [info] exit rotary: 7 (0.27%) [info] exit roundabout turn: 14 (0.53%) [info] (stay on roundabout): 89 (3.38%) [info] Assigned 2631 turn instruction modifiers: [info] uturn: 154 (5.85%) [info] sharp right: 54 (2.05%) [info] right: 541 (20.56%) [info] slight right: 179 (6.80%) [info] straight: 984 (37.40%) [info] slight left: 190 (7.22%) [info] left: 441 (16.76%) [info] sharp left: 88 (3.34%) [info] Guidance turn annotations took 0.084691s [info] Writing Intersection Classification Data [info] ok, after 0.002845s [info] Writing Turns and Lane Data... [info] ok, after 0.004326s [info] Saving edge-based node weights to file. [info] Done writing. (0.001548) [info] Computing strictly connected components ... [info] Found 67 SCC (1 large, 66 small) [info] SCC run took: 6.4058e-05s [info] Building r-tree ... [info] Constructing r-tree of 5046 segments build on-top of 4836 coordinates terminate called after throwing an instance of 'osrm::util::exception' what(): File "/data/monaco-latest.osrm.fileIndex" mapping failed: failed mapping file: Invalid argument: iostream errorinclude/util/mmap_file.hpp:60
prusswan
commented
5 years ago Another workaround is to mount the share dir, but copy the contents to a location within the container, i.e. don't perform I/O within the mounted location. This is useful if the shared contents are not too big in filesize.
Kruszylo
commented
5 years ago Hi, I'm experiencing similar error when I try to run docker container:
docker run --rm -ti -v /home/docker/osm:/osm -p 5000:5000 -p 3000:3000 -e CPUS=4 -e DATA_PATH=/osm -e OSM_PBF_URL=http://download.geofabrik.de/europe/poland/malopolskie-latest.osm.pbf -e SHAREDMEMORY=0 kruszylo/vroom-serverError is:
.
100%
[info] Contracted graph has 1817014 edges.
[info] Contraction took 114.926 sec
[info] Preprocessing : 115.117 seconds
[info] finished preprocessing
[info] RAM: peak bytes used: 193404928
[info] starting up engines, v5.22.0
[info] Threads: 4
[info] IP address: 0.0.0.0
[info] IP port: 5000
terminate called after throwing an instance of 'osrm::util::exception'
what(): File "/osm/osrm/processed/car/malopolskie-latest.osrm.fileIndex" mapping failed: failed opening file: Permission denied: iostream errorinclude/util/mmap_file.hpp:33
2019-07-12 20:27:42,743 INFO exited: osrm (terminated by SIGABRT; not expected)
2019-07-12 20:27:43,746 INFO spawned: 'osrm' with pid 53 Guys, I solved my problem, I needed to make chmod -R 775 /osm* to allow read from compiled files. Docker image should works now (if anybody interested).
denplis
commented
4 years ago workflow with Docker Toolbox Moving berlin-latest.osm.pbf inside image will help.
Dockerfile
FROM osrm/osrm-backend
COPY berlin-latest.osm.pbf /berlin-latest.osm.pbf
RUN osrm-extract -p /opt/car.lua /berlin-latest.osm.pbf
RUN osrm-partition /berlin-latest.osrm
RUN osrm-customize /berlin-latest.osrm
CMD osrm-routed --algorithm mld /berlin-latest.osrmdocker build -t my_osrm .
docker run -t -i -p 5000:5000 my_osrm
OS is Windows 7. Running Docker-Toolkit with boot2docker vm image in Oracle VM VirtualBox 5.2.4. VM is allocated 28GB of memory, 200gb storage, 4 processors.
Downloaded multiple osm.pbf from both Geofabrik and Mapzen. Successfully re-produced on the Mapzen Detroit file and Geofabrik's North America, USA-Michigan, and Berlin files.
Reproducible with both osrm-backend latest docker image and also 5.14.2. Did not try other version.
Files are located in c:\users\public\mapdata so they are visible within Docker quickstart terminal. Run the following commands from within that directory. I tried with all the files listed above and both osrm-backend (current) and the tagged v5.14.2. I'm only showing the current commands below.
$ docker run -t -v $(pwd):/data osrm/osrm-backend osrm-extract -p /opt/car.lua /data/detroit_michigan.osm.pbfGenerated this:
So that looked successful. Following the guide here (https://hub.docker.com/r/osrm/osrm-backend/) I next run:
$ docker run -t -v $(pwd):/data osrm/osrm-backend osrm-partition /data/detroit_michigan.osrmWhich generates the error: