search

motis-project / motis

Intermodal Mobility Information System
https://motis-project.de
MIT License
198 stars 47 forks source link

initialization error: schedule not loaded - when trying to run ./motis #181

Open wkulesza opened 2 years ago

wkulesza commented 2 years ago

Hi, great project, I love the demo. Tried recreating all steps mentioned here https://motis-project.de/docs/install using Switzerland data. I grabbed the code and extracted (directory structure same as suggested) Created data folder Within data folder, I created hrd folder, where i added 4_tu_ch_oev_sammlung_ch__2021_20211201_203455.zip file , within data folder, there's also switzerland-latest.osm.pbf file and log folder, whre txt empty files have been created.

When tring to run motis from root folder i get:

root@vpsxxxxx:/home/user/motis# ./motis
Used Options:

  server.host: 0.0.0.0
  server.port: 8080
  server.cert_path: ::dev::
  server.priv_key_path: ::dev::
  server.dh_path: ::dev::
  server.api_key:
  server.log_path:
  server.static_path:

  import.paths:
  import.data_dir: data
  import.require_successful: 1

  dataset.path: rohdaten
  dataset.prefix:
  dataset.graph_path: default
  dataset.write_serialized: 1
  dataset.write_graph: 0
  dataset.read_graph: 0
  dataset.read_graph_mmap: 0
  dataset.cache_graph: 0
  dataset.apply_rules: 1
  dataset.adjust_footpaths: 1
  dataset.expand_footpaths: 1
  dataset.begin: TODAY
  dataset.num_days: 2
  dataset.planned_transfer_delta: 30
  dataset.wzr_classes_path:
  dataset.wzr_matrix_path:
  dataset.no_local_transport: 0

  modules: address, cc, csa, guesser, intermodal, lookup, osrm, parking, path, paxforecast, paxmon, ppr, railviz, revise, ris, routing, rt, tiles, tripbased
  exclude_modules:

  remotes:

  mode: server
  batch_input_file: queries.txt
  batch_output_file: responses.txt
  init:
  num_threads: 2
  direct: 0

  csa.bridge: 0
  csa.expand_footpaths: 0

  intermodal.router: routing
  intermodal.revise: 0

  osrm.profiles:

  parking.db_max_size: 549755813888
  parking.max_walk_duration: 10
  parking.import.edge_rtree_max_size: 3221225472
  parking.import.area_rtree_max_size: 1073741824
  parking.import.lock_rtrees: 0

  path.use_cache:
  path.max_size: 34359738368

  paxforecast.forecast_results:
  paxforecast.behavior_stats:
  paxforecast.routing_cache:
  paxforecast.calc_load_forecast: 1
  paxforecast.publish_load_forecast: 0
  paxforecast.stats:
  paxforecast.deterministic_mode: 0
  paxforecast.min_delay_improvement: 5

  paxmon.generated_capacity_file:
  paxmon.stats:
  paxmon.capacity_match_log:
  paxmon.journey_match_log:
  paxmon.over_capacity_report:
  paxmon.broken_report:
  paxmon.reroute_unmatched: 0
  paxmon.reroute_file:
  paxmon.reroute_router: /tripbased
  paxmon.start_time: 1970-01-01T00:00:00Z+0000
  paxmon.end_time: 1970-01-01T00:00:00Z+0000
  paxmon.time_step: 60
  paxmon.match_tolerance: 0
  paxmon.arrival_delay_threshold: 20
  paxmon.preparation_time: 15
  paxmon.check_graph_times: 0
  paxmon.check_graph_integrity: 0
  paxmon.mcfp_scenario_dir:
  paxmon.mcfp_scenario_min_broken_groups: 500
  paxmon.mcfp_scenario_include_trip_info: 0
  paxmon.keep_group_history: 0
  paxmon.reuse_groups: 1

  ppr.import.use_dem: 0
  ppr.profile:
  ppr.edge-rtree-max-size: 3221225472
  ppr.area-rtree-max-size: 1073741824
  ppr.lock-rtrees: 0
  ppr.prefetch-rtrees: 1
  ppr.verify-graph: 0

  railviz.initial_permalink:
  railviz.tiles_redirect:

  ris.gtfs_trip_ids:
  ris.db: ris.mdb
  ris.input: ris
  ris.db_max_size: 549755813888
  ris.init_time: 1970-01-01T00:00:00Z+0000
  ris.clear_db: 0
  ris.instant_forward: 0
  ris.gtfsrt.is_addition_skip_allowed: 1

  rt.validate_graph: 0
  rt.validate_constant_graph: 0
  rt.print_stats: 1

  tiles.profile:
  tiles.import.use_coastline: 0
  tiles.import.flush_threshold: 10000000
  tiles.db_size: 1099511627776

  tripbased.use_data_file: 1

     address: WAITING: {"OSM"}
     parking: WAITING: {"OSM", "PPR", "SCHEDULE"}
        path: WAITING: {"OSM", "OSRM", "SCHEDULE"}
      paxmon: WAITING: {"SCHEDULE"}
         ppr: WAITING: {"OSM"}
    schedule: WAITING: {"SCHEDULE"}
       tiles: WAITING: {"OSM"}
   tripbased: WAITING: {"SCHEDULE"}
2021-12-14T13:06:39Z [VERIFY FAIL] schedule not loaded

initialization error: schedule not loaded

Running on Ubuntu 18.04.5 LTS

felixguendling commented 2 years ago

It seems like you are neither passing configuration parameters nor do you have the config.ini file.

I wrote a small setup script for you:

#!/bin/sh

MOTIS_URL=https://github.com/motis-project/motis/releases/latest/download/motis-linux-amd64.tar.bz2
SCHEDULE_URL=https://opentransportdata.swiss/dataset/55b617c1-7241-4254-b243-0352d5523f6f/resource/54262e6d-04df-4d7d-bdb2-b6d173a49d0a/download/4_tu_ch_oev_sammlung_ch__2021_20211208_201836.zip
OSM_URL=https://download.geofabrik.de/europe/switzerland-latest.osm.pbf

# Create folder.
mkdir -p motis-swiss/data/hrd
cd motis-swiss

# Download and unpack MOTIS.
wget $MOTIS_URL
tar -xf motis-linux-amd64.tar.bz2

# Download and unpack timetable.
wget $SCHEDULE_URL
unzip 4_tu_ch_oev_sammlung_ch__2021_20211208_201836.zip -d data/hrd

# Download OSM data
wget -P data $OSM_URL

# Write config.ini
echo "\
server.static_path=motis/web\n\
dataset.begin=20211201\n\
\n\
[import]\n\
paths=schedule:data/hrd\n\
paths=osm:data/switzerland-latest.osm.pbf\n\
data_dir=data\n\
\n\
[osrm]\n\
profiles=motis/osrm-profiles/car.lua\n\
profiles=motis/osrm-profiles/bike.lua\n\
profiles=motis/osrm-profiles/bus.lua\n\
\n\
[ppr]\n\
profile=motis/ppr-profiles/default.json\n\
\n\
[tiles]\n\
profile=motis/tiles-profiles/background.lua\n\
\n" > config.ini

# Start MOTIS
./motis/motis

Please copy this script into a file setup-motis-swiss.sh, make it executable chmod +x setup-motis-swiss.sh and run it ./setup-motis-swiss.sh.

Edit: oops, made a mistake with unpacking the HRD data. Fixed now.

wkulesza commented 2 years ago

Thanks @felixguendling - this script did the trick. I can see Motis running on 8080. Two quick questions:

felixguendling commented 2 years ago

MOTIS can load GTFS and handles GTFS-RT in real-time. Do you have an URL for your feeds?

We run MOTIS detached either via docker compose (see here or via SystemD service unit. I can post our systemd configuration later.

wkulesza commented 2 years ago

Yes, we could start with official feeds for Poznan: newest is here: https://www.ztm.poznan.pl/pl/dla-deweloperow/getGTFSFile/?file=20211214_20211217.zip trip updates are here: https://www.ztm.poznan.pl/pl/dla-deweloperow/getGtfsRtFile/?token=eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJpc3MiOiJ0ZXN0Mi56dG0ucG96bmFuLnBsIiwiY29kZSI6MSwibG9naW4iOiJtaFRvcm8iLCJ0aW1lc3RhbXAiOjE1MTM5NDQ4MTJ9.ND6_VN06FZxRfgVylJghAoKp4zZv6_yZVBu_1-yahlo&file=trip_updates.pb

felixguendling commented 2 years ago

We switched to docker compose for all our MOTIS instances.

But here are the systemd unit files. They belong to /etc/systemd/system.

motis.service

[Unit]
Description=Motis
After=network.target

[Service]
Restart=always
RestartSec=15
TimeoutStartSec=300
User=motis
Group=motis
WorkingDirectory=/home/motis
ExecStart=/home/motis/motis/motis -c /root/motis/config.ini

[Install]
WantedBy=multi-user.target

motis-restart.timer

[Unit]
Description=Restart MOTIS
RequisiteOverridable=motis-restart.service

[Timer]
OnCalendar=*-*-* 03:30:00
Persistent=false

[Install]
WantedBy=timers.target

motis-restart.service

[Unit]
Description=Restart MOTIS

[Service]
Type=oneshot
ExecStart=/bin/systemctl restart motis

Enable timer: systemctl enable motis-restart.timer Enable MOTIS autostart: systemctl enable motis Start MOTIS: systemctl start motis

This files assume that you have user named motis that has a home directory at /home/motis with the usual MOTIS setup.

felixguendling commented 2 years ago

I adapted the script for the Poznan data:

#!/bin/sh

MOTIS_URL=https://github.com/motis-project/motis/releases/latest/download/motis-linux-amd64.tar.bz2
SCHEDULE_URL=https://www.ztm.poznan.pl/pl/dla-deweloperow/getGTFSFile/?file=20211214_20211217.zip
OSM_URL=https://download.bbbike.org/osm/bbbike/Poznan/Poznan.osm.pbf

# Create folder.
mkdir -p motis-poznan/data/gtfs
mkdir -p motis-poznan/data/gtfs-rt
cd motis-poznan

# Download and unpack MOTIS.
wget $MOTIS_URL
tar -xf motis-linux-amd64.tar.bz2

# Download and unpack timetable.
wget $SCHEDULE_URL
unzip index.html?file=20211214_20211217.zip -d data/gtfs
touch data/gtfs/transfers.txt

# Download OSM data
wget -P data $OSM_URL

# Write config.ini
echo "\
modules=intermodal\n\
modules=routing\n\
modules=rt\n\
modules=ris\n\
modules=railviz\n\
modules=path\n\
modules=lookup\n\
modules=address\n\
modules=guesser\n\
modules=tiles\n\
modules=osrm\n\
modules=ppr\n\
modules=parking\n\
\n\
server.static_path=motis/web\n\
dataset.begin=20211214\n\
\n\
[import]\n\
paths=schedule:data/gtfs\n\
paths=osm:data/Poznan.osm.pbf\n\
data_dir=data\n\
\n\
[osrm]\n\
profiles=motis/osrm-profiles/car.lua\n\
profiles=motis/osrm-profiles/bike.lua\n\
profiles=motis/osrm-profiles/bus.lua\n\
\n\
[ppr]\n\
profile=motis/ppr-profiles/default.json\n\
\n\
[tiles]\n\
profile=motis/tiles-profiles/background.lua\n\
\n\
[ris]\n\
gtfsrt.is_addition_skip_allowed=false\n\
instant_forward=true\n\
input=data/gtfs-rt\n\
init_time=NOW\n\
\n" > config.ini

# Start MOTIS
./motis/motis

For GTFS-RT, you need to run this script additionally to MOTIS:

#!/bin/bash
while true; do
  filename="data/gtfs-rt/$(date +%Y%m%d-%H%M%S).pb"
  curl -X GET -H "Content-type: application/octet-stream" -o "$filename" https://www.ztm.poznan.pl/pl/dla-deweloperow/getGtfsRtFile/?token=eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJpc3MiOiJ0ZXN0Mi56dG0ucG96bmFuLnBsIiwiY29kZSI6MSwibG9naW4iOiJtaFRvcm8iLCJ0aW1lc3RhbXAiOjE1MTM5NDQ4MTJ9.ND6_VN06FZxRfgVylJghAoKp4zZv6_yZVBu_1-yahlo&file=trip_updates.pb
  echo "$i" "$filename"
  curl --insecure https://localhost:8080/ris/read
  sleep 30
done
felixguendling commented 2 years ago

The GTFS-RT feed does only contain coordinate updates for the vehicles:

{
  id: "9115"
  trip_update {
    trip {
      trip_id: "1_12179^D"
      schedule_relationship: SCHEDULED
      route_id: "901"
    }
    stop_time_update {
      stop_sequence: 0
      arrival {
        delay: -310
      }
      schedule_relationship: SCHEDULED
    }
    vehicle {
      id: "9115"
      label: "901/2"
    }
    timestamp: 1639509290
  }
}
{
  id: "9102"
  vehicle {
    trip {
      trip_id: "1_12204^D"
      schedule_relationship: SCHEDULED
      route_id: "901"
    }
    position {
      latitude: 52.4793
      longitude: 16.8603096
      speed: 3.06
    }
    current_stop_sequence: 4
    timestamp: 1639509288
    vehicle {
      id: "9102"
      label: "901/3"
    }
  }
}

These are currently not supported by MOTIS.

MOTIS only supports timetable updates like event time changes, reroutings, additional services, etc.

wkulesza commented 2 years ago

@felixguendling great stuff. It works with poznan data. We would be anxious to help out to work on this. Maybe we could do something together on this - have you heard about OneBusAway ? trip updates from GTFS-RT would be real-time info on actual position/ schedule adherence from GTFS timetables - so this could show how does the vehicle actually behave in terms of being ontime/early/late.

felixguendling commented 2 years ago

Always happy to join forces! :sunglasses:

But I'm not exactly sure what you're proposing. Feel free to reach out to me guendling@cs.tu-darmstadt.de.

danielkarch commented 2 years ago

Hello Felix,

I just stumbled upon Motis and tried to run it using your setup-motis-swiss.sh script above. It runs for a while but then fails with 

     address: FINISHED
   osrm-bike: FINISHED
    osrm-bus: FINISHED
    osrm-car: FINISHED
     parking: WAITING: {"PPR", "SCHEDULE"}
        path: WAITING: {"OSRM", "SCHEDULE"}
      paxmon: WAITING: {"SCHEDULE"}
         ppr: [■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■         ]  84% | Graph Serialization
    schedule: FINISHED
       tiles: WAITING: {"OSM"}
   tripbased: WAITING: {"SCHEDULE"}
fish: Job 1, './motis/motis' terminated by signal SIGBUS (Misaligned address error)

Do you know why this might be? This is on a 64-bit Linux system.

felixguendling commented 2 years ago

This could be caused by running AVX instructions on a (potentially virtualized) CPU that doesn't support AVX. Maybe we could eliminate this problem source by using this binary for now: https://github.com/motis-project/motis/releases/download/v0.6.3/motis-linux-amd64-noavx.tar.bz2

Would be happy to hear whether that helps! 😃

danielkarch commented 2 years ago

It appears that I was just running out of disk space. Now I have it running. Thanks for your help!