GIScience / openrouteservice

🌍 The open source route planner api with plenty of features.
https://openrouteservice.org
GNU General Public License v3.0
1.36k stars 386 forks source link

Not enough space - Import Error #1088

Closed jpylypiw closed 2 years ago

jpylypiw commented 2 years ago

Here's what I did

I am using the docker image on a rather large hardware system.

Specs: Intel® Core™ i7-8700 Hexa-Core 128GB DDR4 RAM 2x 1TB NVMe SSD Gigabit Ethernet

My system is managed by ansible so I can not use the docker compose directly. This is my ansible code:

vars:
    java_xms: "60g"
    java_xmx: "175g"
    ors_version: "latest"

    - name: "create ors-app container"
      community.general.docker_container:
        name: "ors-app"
        image: "openrouteservice/openrouteservice:{{ ors_version }}"
        restart_policy: "on-failure"
        container_default_behavior: "compatibility"
        pull: true
        state: started
        # tty: true
        ports:
          - "127.0.0.1:8080:8080"
        volumes:
          - /opt/ors-app/graphs:/ors-core/data/graphs
          - /opt/ors-app/elevation_cache:/ors-core/data/elevation_cache
          - /opt/ors-app/ors-conf:/ors-conf
          - /opt/ors-app/ors-log:/var/log/ors
          - /opt/ors-app/tomcat-log:/usr/local/tomcat/logs
          - /opt/ors-app/pbf/planet-latest.osm.pbf:/ors-core/data/osm_file.pbf
        env:
          JAVA_OPTS: "-Djava.awt.headless=true -server -XX:TargetSurvivorRatio=75 -XX:SurvivorRatio=64 -XX:MaxTenuringThreshold=3 -XX:+UseG1GC -XX:+ScavengeBeforeFullGC -XX:ParallelGCThreads=4 -Xms{{ java_xms }} -Xmx{{ java_xmx }}"
          CATALINA_OPTS: "-Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.port=9001 -Dcom.sun.management.jmxremote.rmi.port=9001 -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.ssl=false -Djava.rmi.server.hostname=localhost"
          BUILD_GRAPHS: "True"

As OSM File I use the Planet OSM from bbbike.org since it does not contain any meta data, which shrinks the file a lot. The file is only 48GB instead of 63GB of the whole planet. I hope this speeds up the import process.

planet_osm_nometa_url: "https://download.bbbike.org/osm/planet/planet-latest-nometa.osm.pbf"
planet_osm_nometa_checksum_url: "https://download.bbbike.org/osm/planet/planet-latest-nometa.osm.pbf.md5"

I also use a custom ors-config.json file which I show you here:

{
    "ors": {
        "info": {
            "base_url": "https://openrouteservice.org/",
            "support_mail": "support@openrouteservice.org",
            "author_tag": "openrouteservice",
            "content_licence": "LGPL 3.0"
        },
        "services": {
            "matrix": {
                "enabled": true,
                "maximum_routes": 100,
                "maximum_routes_flexible": 25,
                "maximum_search_radius": 5000,
                "maximum_visited_nodes": 100000,
                "allow_resolve_locations": true,
                "attribution": "openrouteservice.org, OpenStreetMap contributors"
            },
            "isochrones": {
                "enabled": false,
                "maximum_range_distance": [
                    {
                        "profiles": "any",
                        "value": 50000
                    },
                    {
                        "profiles": "driving-car, driving-hgv",
                        "value": 100000
                    }
                ],
                "maximum_range_time": [
                    {
                        "profiles": "any",
                        "value": 18000
                    },
                    {
                        "profiles": "driving-car, driving-hgv",
                        "value": 3600
                    }
                ],
                "fastisochrones": {
                    "maximum_range_distance": [
                        {
                            "profiles": "any",
                            "value": 50000
                        },
                        {
                            "profiles": "driving-car, driving-hgv",
                            "value": 500000
                        }
                    ],
                    "maximum_range_time": [
                        {
                            "profiles": "any",
                            "value": 18000
                        },
                        {
                            "profiles": "driving-car, driving-hgv",
                            "value": 10800
                        }
                    ],
                    "profiles": {
                        "default_params": {
                            "enabled": false,
                            "threads": 12,
                            "weightings": "recommended",
                            "maxcellnodes": 5000
                        },
                        "profile-hgv": {
                            "enabled": true,
                            "threads": 12,
                            "weightings": "recommended, shortest",
                            "maxcellnodes": 5000
                        }
                    }
                },
                "maximum_intervals": 10,
                "maximum_locations": 2,
                "allow_compute_area": true
            },
            "routing": {
                "enabled": true,
                "mode": "normal",
                "routing_description": "This is a routing file from openrouteservice",
                "routing_name": "openrouteservice routing",
                "sources": [
                    "data/osm_file.pbf"
                ],
                "init_threads": 1,
                "attribution": "openrouteservice.org, OpenStreetMap contributors",
                "elevation_preprocessed": false,
                "profiles": {
                    "active": [
                        "car"
                    ],
                    "default_params": {
                        "encoder_flags_size": 8,
                        "graphs_root_path": "data/graphs",
                        "elevation_provider": "multi",
                        "elevation_cache_path": "data/elevation_cache",
                        "elevation_cache_clear": false,
                        "instructions": true,
                        "maximum_distance": 1000000,
                        "maximum_distance_dynamic_weights": 100000,
                        "maximum_distance_avoid_areas": 100000,
                        "maximum_waypoints": 50,
                        "maximum_snapping_radius": 400,
                        "maximum_avoid_polygon_area": 200000000,
                        "maximum_avoid_polygon_extent": 20000,
                        "maximum_distance_alternative_routes": 100000,
                        "maximum_alternative_routes": 3,
                        "maximum_distance_round_trip_routes": 100000,
                        "maximum_speed_lower_bound": 80,
                        "preparation": {
                            "min_network_size": 200,
                            "min_one_way_network_size": 200,
                            "methods": {
                                "lm": {
                                    "enabled": true,
                                    "threads": 1,
                                    "weightings": "recommended,shortest",
                                    "landmarks": 16
                                }
                            }
                        },
                        "execution": {
                            "methods": {
                                "lm": {
                                    "disabling_allowed": true,
                                    "active_landmarks": 8
                                }
                            }
                        }
                    },
                    "profile-car": {
                        "profiles": "driving-car",
                        "parameters": {
                            "encoder_flags_size": 8,
                            "encoder_options": "turn_costs=true|block_fords=false|use_acceleration=true",
                            "maximum_distance": 1000000,
                            "elevation": true,
                            "maximum_snapping_radius": 350,
                            "preparation": {
                                "min_network_size": 200,
                                "min_one_way_network_size": 200,
                                "methods": {
                                    "ch": {
                                        "enabled": true,
                                        "threads": 1,
                                        "weightings": "fastest"
                                    },
                                    "lm": {
                                        "enabled": false,
                                        "threads": 1,
                                        "weightings": "fastest,shortest",
                                        "landmarks": 16
                                    },
                                    "core": {
                                        "enabled": true,
                                        "threads": 1,
                                        "weightings": "fastest,shortest",
                                        "landmarks": 64,
                                        "lmsets": "highways;allow_all"
                                    }
                                }
                            },
                            "execution": {
                                "methods": {
                                    "ch": {
                                        "disabling_allowed": true
                                    },
                                    "lm": {
                                        "disabling_allowed": true,
                                        "active_landmarks": 6
                                    },
                                    "core": {
                                        "disabling_allowed": true,
                                        "active_landmarks": 6
                                    }
                                }
                            },
                            "ext_storages": {
                                "WayCategory": {},
                                "HeavyVehicle": {},
                                "WaySurfaceType": {},
                                "RoadAccessRestrictions": {
                                    "use_for_warnings": true
                                }
                            }
                        }
                    },
                    "profile-hgv": {
                        "profiles": "driving-hgv",
                        "parameters": {
                            "encoder_flags_size": 8,
                            "encoder_options": "turn_costs=true|block_fords=false|use_acceleration=true",
                            "maximum_distance": 1000000,
                            "elevation": true,
                            "preparation": {
                                "min_network_size": 200,
                                "min_one_way_network_size": 200,
                                "methods": {
                                    "ch": {
                                        "enabled": true,
                                        "threads": 1,
                                        "weightings": "recommended"
                                    },
                                    "core": {
                                        "enabled": true,
                                        "threads": 1,
                                        "weightings": "recommended,shortest",
                                        "landmarks": 64,
                                        "lmsets": "highways;allow_all"
                                    }
                                }
                            },
                            "execution": {
                                "methods": {
                                    "ch": {
                                        "disabling_allowed": true
                                    },
                                    "core": {
                                        "disabling_allowed": true,
                                        "active_landmarks": 6
                                    }
                                }
                            },
                            "ext_storages": {
                                "WayCategory": {},
                                "HeavyVehicle": {
                                    "restrictions": true
                                },
                                "WaySurfaceType": {}
                            }
                        }
                    }
                }
            }
        },
        "logging": {
            "enabled": true,
            "level_file": "DEBUG_LOGGING.json",
            "location": "/var/log/ors",
            "stdout": true
        },
        "system_message": []
    }
}

There is only one profile active which is "driving-car".


Here's what I got

The import was running rather smooth since it created the elevation cache files just in a few hours. After a few hours I got a error:

OpenJDK 64-Bit Server VM warning: INFO: os::commit_memory(0x00007caac19ed000, 16384, 0) failed; error='Not enough space' (errno=12)
#
# There is insufficient memory for the Java Runtime Environment to continue.
# Native memory allocation (mmap) failed to map 16384 bytes for committing reserved memory.
# An error report file with more information is saved as:
# /ors-core/hs_err_pid128.log

This is the whole log:

root@hostname ~ # docker logs -f ors-app
### openrouteservice configuration ###
ors-config.json exists in ors-conf folder. Copy config to /ors-core/openrouteservice/src/main/resources/ors-config.json
### Package openrouteservice and deploy to Tomcat ###
WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by com.google.inject.internal.cglib.core.$ReflectUtils$1 (file:/usr/share/maven/lib/guice.jar) to method java.lang.ClassLoader.defineClass(java.lang.String,byte[],int,int,java.security.ProtectionDomain)
WARNING: Please consider reporting this to the maintainers of com.google.inject.internal.cglib.core.$ReflectUtils$1
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release
NOTE: Picked up JDK_JAVA_OPTIONS:  --add-opens=java.base/java.lang=ALL-UNNAMED --add-opens=java.base/java.io=ALL-UNNAMED --add-opens=java.rmi/sun.rmi.transport=ALL-UNNAMED
24-Nov-2021 08:39:26.872 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Server version:        Apache Tomcat/8.5.39
24-Nov-2021 08:39:26.874 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Server built:          Mar 14 2019 11:24:26 UTC
24-Nov-2021 08:39:26.874 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Server number:         8.5.39.0
24-Nov-2021 08:39:26.874 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log OS Name:               Linux
24-Nov-2021 08:39:26.874 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log OS Version:            5.10.0-9-amd64
24-Nov-2021 08:39:26.874 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Architecture:          amd64
24-Nov-2021 08:39:26.874 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Java Home:             /usr/local/openjdk-11
24-Nov-2021 08:39:26.874 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log JVM Version:           11.0.12+7
24-Nov-2021 08:39:26.874 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log JVM Vendor:            Oracle Corporation
24-Nov-2021 08:39:26.874 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log CATALINA_BASE:         /usr/local/tomcat
24-Nov-2021 08:39:26.874 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log CATALINA_HOME:         /usr/local/tomcat
24-Nov-2021 08:39:26.875 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: --add-opens=java.base/java.lang=ALL-UNNAMED
24-Nov-2021 08:39:26.875 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: --add-opens=java.base/java.io=ALL-UNNAMED
24-Nov-2021 08:39:26.875 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: --add-opens=java.rmi/sun.rmi.transport=ALL-UNNAMED
24-Nov-2021 08:39:26.875 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Djava.util.logging.config.file=/usr/local/tomcat/conf/logging.properties
24-Nov-2021 08:39:26.875 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager
24-Nov-2021 08:39:26.875 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Djava.awt.headless=true
24-Nov-2021 08:39:26.875 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -XX:TargetSurvivorRatio=75
24-Nov-2021 08:39:26.876 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -XX:SurvivorRatio=64
24-Nov-2021 08:39:26.876 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -XX:MaxTenuringThreshold=3
24-Nov-2021 08:39:26.876 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -XX:+UseG1GC
24-Nov-2021 08:39:26.876 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -XX:+ScavengeBeforeFullGC
24-Nov-2021 08:39:26.876 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -XX:ParallelGCThreads=4
24-Nov-2021 08:39:26.876 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Xms60g
24-Nov-2021 08:39:26.876 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Xmx150g
24-Nov-2021 08:39:26.876 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Djdk.tls.ephemeralDHKeySize=2048
24-Nov-2021 08:39:26.876 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Djava.protocol.handler.pkgs=org.apache.catalina.webresources
24-Nov-2021 08:39:26.876 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Dorg.apache.catalina.security.SecurityListener.UMASK=0027
24-Nov-2021 08:39:26.876 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Dcom.sun.management.jmxremote
24-Nov-2021 08:39:26.876 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Dcom.sun.management.jmxremote.port=9001
24-Nov-2021 08:39:26.876 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Dcom.sun.management.jmxremote.rmi.port=9001
24-Nov-2021 08:39:26.877 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Dcom.sun.management.jmxremote.authenticate=false
24-Nov-2021 08:39:26.877 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Dcom.sun.management.jmxremote.ssl=false
24-Nov-2021 08:39:26.877 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Djava.rmi.server.hostname=localhost
24-Nov-2021 08:39:26.877 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Dignore.endorsed.dirs=
24-Nov-2021 08:39:26.877 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Dcatalina.base=/usr/local/tomcat
24-Nov-2021 08:39:26.877 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Dcatalina.home=/usr/local/tomcat
24-Nov-2021 08:39:26.877 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Djava.io.tmpdir=/usr/local/tomcat/temp
24-Nov-2021 08:39:26.877 INFO [main] org.apache.catalina.core.AprLifecycleListener.lifecycleEvent The APR based Apache Tomcat Native library which allows optimal performance in production environments was not found on the java.library.path: [/usr/java/packages/lib:/usr/lib64:/lib64:/lib:/usr/lib]
24-Nov-2021 08:39:26.899 INFO [main] org.apache.coyote.AbstractProtocol.init Initializing ProtocolHandler ["http-nio-8080"]
24-Nov-2021 08:39:26.905 INFO [main] org.apache.tomcat.util.net.NioSelectorPool.getSharedSelector Using a shared selector for servlet write/read
24-Nov-2021 08:39:26.914 INFO [main] org.apache.coyote.AbstractProtocol.init Initializing ProtocolHandler ["ajp-nio-8009"]
24-Nov-2021 08:39:26.915 INFO [main] org.apache.tomcat.util.net.NioSelectorPool.getSharedSelector Using a shared selector for servlet write/read
24-Nov-2021 08:39:26.915 INFO [main] org.apache.catalina.startup.Catalina.load Initialization processed in 253 ms
24-Nov-2021 08:39:26.949 INFO [main] org.apache.catalina.core.StandardService.startInternal Starting service [Catalina]
24-Nov-2021 08:39:26.950 INFO [main] org.apache.catalina.core.StandardEngine.startInternal Starting Servlet Engine: Apache Tomcat/8.5.39
24-Nov-2021 08:39:26.966 INFO [localhost-startStop-1] org.apache.catalina.startup.HostConfig.deployWAR Deploying web application archive [/usr/local/tomcat/webapps/ors.war]
08:39:31.053 [localhost-startStop-1] INFO  org.heigit.ors.config.AppConfig - Default path of 'ors-config.json' used for configuration
08:39:31.060 [localhost-startStop-1] INFO  org.heigit.ors.config.AppConfig - Loading configuration from /usr/local/tomcat/webapps/ors/WEB-INF/classes/ors-config.json

  .   ____          _            __ _ _
 /\\ / ___'_ __ _ _(_)_ __  __ _ \ \ \ \
( ( )\___ | '_ | '_| | '_ \/ _` | \ \ \ \
 \\/  ___)| |_)| | | | | || (_| |  ) ) ) )
  '  |____| .__|_| |_|_| |_\__, | / / / /
 =========|_|==============|___/=/_/_/_/
 :: Spring Boot ::        (v2.3.5.RELEASE)

24 Nov 08:39:31 INFO [ors.Application] - Starting Application v6.6.1 on ec143718555a with PID 128 (/usr/local/tomcat/webapps/ors/WEB-INF/classes started by ors in /ors-core)
24 Nov 08:39:31 DEBUG [ors.Application] - Running with Spring Boot v2.3.5.RELEASE, Spring v5.2.10.RELEASE
24 Nov 08:39:31 INFO [ors.Application] - No active profile set, falling back to default profiles: default
SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
24 Nov 08:39:33 INFO [ors.Application] - Started Application in 2.852 seconds (JVM running for 7.417)
24-Nov-2021 08:39:33.789 INFO [localhost-startStop-1] org.apache.jasper.servlet.TldScanner.scanJars At least one JAR was scanned for TLDs yet contained no TLDs. Enable debug logging for this logger for a complete list of JARs that were scanned but no TLDs were found in them. Skipping unneeded JARs during scanning can improve startup time and JSP compilation time.
<?xml version="1.0" ?>
<Configuration>
  <Properties>
    <Property name="filename">/var/log/ors/ors-logs.log</Property>
  </Properties>
</Configuration>
24 Nov 08:39:33 INFO [routing.RoutingProfileManager] -  Total - 60 GB, Free - 59.45 GB, Max: 150 GB, Used - 563.71 MB
24 Nov 08:39:33 INFO [routing.RoutingProfileManager] -
24-Nov-2021 08:39:33.823 INFO [localhost-startStop-1] org.apache.catalina.startup.HostConfig.deployWAR Deployment of web application archive [/usr/local/tomcat/webapps/ors.war] has finished in [6,856] ms
24-Nov-2021 08:39:33.823 INFO [localhost-startStop-1] org.apache.catalina.startup.HostConfig.deployDirectory Deploying web application directory [/usr/local/tomcat/webapps/manager]
24 Nov 08:39:33 INFO [routing.RoutingProfileManager] - ====> Initializing profiles from 'data/osm_file.pbf' (1 threads) ...
24 Nov 08:39:33 INFO [routing.RoutingProfileManager] -
24 Nov 08:39:33 INFO [routing.RoutingProfileManager] -
24-Nov-2021 08:39:33.845 INFO [localhost-startStop-1] org.apache.catalina.startup.HostConfig.deployDirectory Deployment of web application directory [/usr/local/tomcat/webapps/manager] has finished in [22] ms
24-Nov-2021 08:39:33.846 INFO [localhost-startStop-1] org.apache.catalina.startup.HostConfig.deployDirectory Deploying web application directory [/usr/local/tomcat/webapps/host-manager]
24 Nov 08:39:33 INFO [routing.RoutingProfile] - [1] Profiles: 'driving-car', location: 'data/graphs/car'.
24-Nov-2021 08:39:33.858 INFO [localhost-startStop-1] org.apache.catalina.startup.HostConfig.deployDirectory Deployment of web application directory [/usr/local/tomcat/webapps/host-manager] has finished in [13] ms
24-Nov-2021 08:39:33.858 INFO [localhost-startStop-1] org.apache.catalina.startup.HostConfig.deployDirectory Deploying web application directory [/usr/local/tomcat/webapps/examples]
24-Nov-2021 08:39:33.973 INFO [localhost-startStop-1] org.apache.catalina.startup.HostConfig.deployDirectory Deployment of web application directory [/usr/local/tomcat/webapps/examples] has finished in [115] ms
24-Nov-2021 08:39:33.973 INFO [localhost-startStop-1] org.apache.catalina.startup.HostConfig.deployDirectory Deploying web application directory [/usr/local/tomcat/webapps/docs]
24-Nov-2021 08:39:33.982 INFO [localhost-startStop-1] org.apache.catalina.startup.HostConfig.deployDirectory Deployment of web application directory [/usr/local/tomcat/webapps/docs] has finished in [9] ms
24-Nov-2021 08:39:33.982 INFO [localhost-startStop-1] org.apache.catalina.startup.HostConfig.deployDirectory Deploying web application directory [/usr/local/tomcat/webapps/ROOT]
24-Nov-2021 08:39:33.992 INFO [localhost-startStop-1] org.apache.catalina.startup.HostConfig.deployDirectory Deployment of web application directory [/usr/local/tomcat/webapps/ROOT] has finished in [10] ms
24-Nov-2021 08:39:33.995 INFO [main] org.apache.coyote.AbstractProtocol.start Starting ProtocolHandler ["http-nio-8080"]
24-Nov-2021 08:39:34.000 INFO [main] org.apache.coyote.AbstractProtocol.start Starting ProtocolHandler ["ajp-nio-8009"]
24-Nov-2021 08:39:34.001 INFO [main] org.apache.catalina.startup.Catalina.start Server startup in 7086 ms
24 Nov 11:46:05 WARN [servlet.PageNotFound] - Request method 'GET' not supported
24-Nov-2021 13:14:06.031 WARNING [ORS-pl-car] org.heigit.ors.routing.graphhopper.extensions.GraphProcessContext.processWay null. Way id = 14441449
24-Nov-2021 13:26:33.551 WARNING [ORS-pl-car] org.heigit.ors.routing.graphhopper.extensions.GraphProcessContext.processWay null. Way id = 140787316
24-Nov-2021 13:26:44.643 WARNING [ORS-pl-car] org.heigit.ors.routing.graphhopper.extensions.GraphProcessContext.processWay null. Way id = 144082390
24-Nov-2021 13:26:44.643 WARNING [ORS-pl-car] org.heigit.ors.routing.graphhopper.extensions.GraphProcessContext.processWay null. Way id = 144082392
24-Nov-2021 13:26:44.644 WARNING [ORS-pl-car] org.heigit.ors.routing.graphhopper.extensions.GraphProcessContext.processWay null. Way id = 144082396
24-Nov-2021 13:39:28.121 WARNING [ORS-pl-car] org.heigit.ors.routing.graphhopper.extensions.GraphProcessContext.processWay null. Way id = 344910803
24-Nov-2021 13:40:28.614 WARNING [ORS-pl-car] org.heigit.ors.routing.graphhopper.extensions.GraphProcessContext.processWay null. Way id = 367921662
24-Nov-2021 13:44:12.792 WARNING [ORS-pl-car] org.heigit.ors.routing.graphhopper.extensions.GraphProcessContext.processWay null. Way id = 444671739
24-Nov-2021 13:44:13.122 WARNING [ORS-pl-car] org.heigit.ors.routing.graphhopper.extensions.GraphProcessContext.processWay null. Way id = 444772492
24-Nov-2021 13:44:49.229 WARNING [ORS-pl-car] org.heigit.ors.routing.graphhopper.extensions.GraphProcessContext.processWay null. Way id = 459126966
24-Nov-2021 13:46:06.521 WARNING [ORS-pl-car] org.heigit.ors.routing.graphhopper.extensions.GraphProcessContext.processWay null. Way id = 489170464
24-Nov-2021 13:48:06.781 WARNING [ORS-pl-car] org.heigit.ors.routing.graphhopper.extensions.GraphProcessContext.processWay null. Way id = 538099774
24-Nov-2021 13:48:25.450 WARNING [ORS-pl-car] org.heigit.ors.routing.graphhopper.extensions.GraphProcessContext.processWay null. Way id = 543991865
24-Nov-2021 13:50:00.650 WARNING [ORS-pl-car] org.heigit.ors.routing.graphhopper.extensions.GraphProcessContext.processWay null. Way id = 576931605
24-Nov-2021 13:52:18.928 WARNING [ORS-pl-car] org.heigit.ors.routing.graphhopper.extensions.GraphProcessContext.processWay null. Way id = 628798571
24-Nov-2021 13:52:19.007 WARNING [ORS-pl-car] org.heigit.ors.routing.graphhopper.extensions.GraphProcessContext.processWay null. Way id = 628818394
24-Nov-2021 13:53:59.290 WARNING [ORS-pl-car] org.heigit.ors.routing.graphhopper.extensions.GraphProcessContext.processWay null. Way id = 667270499
OpenJDK 64-Bit Server VM warning: INFO: os::commit_memory(0x00007caac19ed000, 16384, 0) failed; error='Not enough space' (errno=12)
#
# There is insufficient memory for the Java Runtime Environment to continue.
# Native memory allocation (mmap) failed to map 16384 bytes for committing reserved memory.
# An error report file with more information is saved as:
# /ors-core/hs_err_pid128.log

Here's what I was expecting

Since the RAM in XMX is more than double the size of the pbf file I thought the import would work fine. The system had more RAM and SWAP ready, so I also checked the OOM killer, but there was no action by the OOM killer. I am kinda speachless since I have no idea how I can solve this issue. I need this working for a customer. Do you have any idea what I can do to solve the issue?


Here's what I think could be improved

rabidllama commented 2 years ago

Hi @jpylypiw,

this is most likely due to needing to increase the memory mapping count. For our machines that run the planet builds, we add the following to the /etc/sysctl.conf file:

vm.max_map_count=81920

jpylypiw commented 2 years ago

Hi @rabidllama, thank you for the fast reply! I increased the value this morning and the container is running since six hours without any issue. I will keep you updated if this fixed the issue but currently it is looking very good.