Deployment

[ghost]

hi @alexlanz thanks for your help, if you can to merge this PR, we can take the changes and further improve the documentation.

I was wondering is that maybe two .env.example files are needed for this project (one for building and another for production)

[alexlanz]

Hi @stemove, I will continue working on the deployment tomorrow. So there can occur changes. But if you want you can merge everything and I will make the last small changes on the development branch directly. And you could open an PR and work from there. What do you think?

Regarding the .env.example file it's true! We need two for the two use cases. But the .env.example file is just a starting point that should indicate all necessary variables. The final .env file will be created inside the Jenkins job.

[rcavaliere]

@alexlanz what is the situation with the deployment? Do we have a testing system of OTP running? Is there an URL that we can have in order to test the basic UI which is made available with the basic installation of OTP?

[alexlanz]

Hi @rcavaliere, the application is now online: https://otp-graph.opendatahub.testingmachine.eu/

But the build server is not starting/stoping automatically yet. And also the builds are not scheduled automatically. I will tackle this point next week.

[rcavaliere]

Hi @alexlanz nice! It's normal that I don't see any data in it, @stemove ? Is there anything more that should be configured?

[zabuTNT]

Hi @rcavaliere. When this happen it means that Otp is running but without a graph. Maybe the path is wrong?

[alexlanz]

I check the path and there seems everything fine:

root@ip-172-31-11-34:/opt/odh-mentor-otp# ls
10_397045932724035_45_94607517243942_10_91385821636202_46_30580331792924.osm   10_91385821636202_45_94607517243942_11_430670500000003_46_30580331792924.osm   11_430670500000003_45_94607517243942_11_947482783637987_46_30580331792924.osm   11_947482783637987_46_30580331792924_12_464295067275971_46_66553146341906.osm   12_464295067275971_46_66553146341906_12_981107350913955_47_025259608908875.osm  osm.url
10_397045932724035_46_30580331792924_10_91385821636202_46_66553146341906.osm   10_91385821636202_46_30580331792924_11_430670500000003_46_66553146341906.osm   11_430670500000003_46_30580331792924_11_947482783637987_46_66553146341906.osm   11_947482783637987_46_66553146341906_12_464295067275971_47_025259608908875.osm  9_880233649086051_46_30580331792924_10_397045932724035_46_66553146341906.osm    srtm_39_03.tif
10_397045932724035_46_66553146341906_10_91385821636202_47_025259608908875.osm  10_91385821636202_46_66553146341906_11_430670500000003_47_025259608908875.osm  11_430670500000003_46_66553146341906_11_947482783637987_47_025259608908875.osm  11_947482783637987_47_025259608908875_12_464295067275971_47_38498775439869.osm  9_880233649086051_46_66553146341906_10_397045932724035_47_025259608908875.osm
10_397045932724035_47_025259608908875_10_91385821636202_47_38498775439869.osm  10_91385821636202_47_025259608908875_11_430670500000003_47_38498775439869.osm  11_430670500000003_47_025259608908875_11_947482783637987_47_38498775439869.osm  12_464295067275971_46_30580331792924_12_981107350913955_46_66553146341906.osm   openmove

But there are errors in the logs:

root@ip-172-31-11-34:/var/docker/odh-mentor-otp-run/current# docker-compose logs -f
Attaching to odh-mentor-otp-run_app_1
app_1  | 14:09:04.831 INFO (OTPServer.java:39) Wiring up and configuring server.
app_1  | 14:09:04.840 INFO (GraphScanner.java:64) Attempting to automatically register routerIds [openmove]
app_1  | 14:09:04.840 INFO (GraphScanner.java:65) Graph files will be sought in paths relative to /data
app_1  | 14:09:04.844 INFO (GraphService.java:176) Registering new router 'openmove'
app_1  | 14:09:04.862 INFO (InputStreamGraphSource.java:181) Loading graph...
app_1  | 14:09:05.519 INFO (Graph.java:746) Graph version: MavenVersion(1, 4, 0, , b272f14007c97d769216e9ebab65baad7410cdf5)
app_1  | 14:09:05.519 INFO (Graph.java:747) OTP version:   MavenVersion(1, 4, 0, , b272f14007c97d769216e9ebab65baad7410cdf5)
app_1  | 14:09:05.519 INFO (Graph.java:764) This graph was built with the currently running version and commit of OTP.
app_1  | 14:09:11.166 INFO (Graph.java:731) Main graph read. |V|=189887 |E|=460955
app_1  | 14:09:19.527 INFO (GraphIndex.java:127) Indexing graph...
app_1  | 14:09:20.500 INFO (GraphIndex.java:595) Clustering stops by geographic proximity and name...
app_1  | 14:09:20.505 INFO (GraphIndex.java:210) Creating a spatial index for stop clusters.
app_1  | 14:09:20.571 INFO (GraphIndex.java:192) Initializing areas....
app_1  | 14:09:20.572 INFO (GraphIndex.java:199) Done indexing graph.
app_1  | 14:09:20.847 INFO (Router.java:100) No default routing parameters were found in the router config JSON. Using built-in OTP defaults.
app_1  | 14:09:20.854 INFO (Router.java:127) Timeouts for router 'openmove': [5.0, 4.0, 2.0]
app_1  | 14:09:20.854 INFO (Router.java:134) Incoming requests will not be logged.
app_1  | 14:09:20.855 INFO (GraphUpdaterConfigurator.java:41) Using configurations: [main]
app_1  | 14:09:21.166 INFO (Router.java:172) Computed ellipsoid/geoid offset at (46.4749269, 11.435274249999999) as 50.66475352457623
app_1  | 14:09:21.178 INFO (GrizzlyServer.java:72) Starting OTP Grizzly server on ports 8080 (HTTP) and 8081 (HTTPS) of interface 0.0.0.0
app_1  | 14:09:21.179 INFO (GrizzlyServer.java:74) OTP server base path is /var/otp
app_1  | 14:09:21.228 INFO (GrizzlyServer.java:51) Java reports that this machine has 2 available processors.
app_1  | 14:09:21.229 INFO (GrizzlyServer.java:62) Maximum HTTP handler thread pool size will be 4 threads.
app_1  | 14:09:21.338 INFO (ServiceFinder.java:172) Running in a non-OSGi environment
app_1  | 14:09:21.939 INFO (ApplicationHandler.java:419) Initiating Jersey application, version Jersey: 2.18 2015-06-05 02:28:21...
app_1  | 14:09:26.132 INFO (ExtendedLogger.java:345) Selected ExecutorServiceProvider implementation [org.glassfish.jersey.server.internal.process.ServerProcessingBinder$DefaultManagedAsyncExecutorProvider] to be used for injection of executor qualified by [org.glassfish.jersey.server.ManagedAsyncExecutor] annotation.
app_1  | 14:09:26.154 INFO (ExtendedLogger.java:345) Selected ScheduledExecutorServiceProvider implementation [org.glassfish.jersey.server.internal.process.ServerProcessingBinder$DefaultBackgroundSchedulerProvider] to be used for injection of scheduler qualified by [org.glassfish.jersey.server.BackgroundScheduler] annotation.
app_1  | 14:09:26.535 WARN (PropertiesHelper.java:330) There is no way how to transform value "true" [java.lang.Boolean] to type [java.lang.String].
app_1  | 14:09:26.539 INFO (ApplicationHandler.java:683) Jersey application initialized.
app_1  | Root Resource Classes:
app_1  |   org.opentripplanner.api.resource.SimpleIsochrone
app_1  |   org.opentripplanner.api.resource.SIsochrone
app_1  |   org.opentripplanner.api.resource.PlannerResource
app_1  |   org.opentripplanner.api.resource.TimeGridWs
app_1  |   org.opentripplanner.index.GeocoderResource
app_1  |   org.opentripplanner.api.resource.ScriptResource
app_1  |   org.opentripplanner.api.resource.ServerInfo
app_1  |   org.opentripplanner.api.resource.SurfaceResource
app_1  |   org.opentripplanner.api.resource.RepeatedRaptorTestResource
app_1  |   org.opentripplanner.api.resource.AlertPatcher
app_1  |   org.opentripplanner.api.resource.ExternalGeocoderResource
app_1  |   org.opentripplanner.api.resource.GraphInspectorTileResource
app_1  |   org.opentripplanner.api.resource.UpdaterStatusResource
app_1  |   org.opentripplanner.api.resource.Routers
app_1  |   org.opentripplanner.api.resource.LegendResource
app_1  |   org.opentripplanner.api.resource.ProfileResource
app_1  |   org.opentripplanner.api.resource.PointSetResource
app_1  |   org.opentripplanner.api.resource.LIsochrone
app_1  |   org.opentripplanner.api.resource.BikeRental
app_1  |   org.opentripplanner.index.IndexAPI
app_1  |   org.opentripplanner.api.resource.ScenarioResource
app_1  |   org.opentripplanner.api.resource.TileService
app_1  | Global Request Filters:
app_1  |    org.opentripplanner.standalone.AuthFilter
app_1  |    org.opentripplanner.standalone.CorsFilter
app_1  | Global Response Filters:
app_1  |    org.opentripplanner.standalone.CorsFilter
app_1  | Global Reader Interceptors:
app_1  |    org.glassfish.jersey.server.internal.MappableExceptionWrapperInterceptor
app_1  | Global Writer Interceptors:
app_1  |    org.glassfish.jersey.server.internal.MappableExceptionWrapperInterceptor
app_1  |    org.glassfish.jersey.server.internal.JsonWithPaddingInterceptor
app_1  | Dynamic Features:
app_1  |    org.glassfish.jersey.server.filter.RolesAllowedDynamicFeature@167279d1
app_1  | Message Body Readers:
app_1  |    org.glassfish.jersey.message.internal.FormProvider
app_1  |    org.glassfish.jersey.jaxb.internal.XmlJaxbElementProvider$Text
app_1  |    org.glassfish.jersey.media.multipart.internal.MultiPartReaderServerSide
app_1  |    org.glassfish.jersey.jaxb.internal.XmlCollectionJaxbProvider$General
app_1  |    org.glassfish.jersey.jaxb.internal.XmlCollectionJaxbProvider$Text
app_1  |    org.glassfish.jersey.message.internal.ByteArrayProvider
app_1  |    org.glassfish.jersey.jaxb.internal.XmlRootObjectJaxbProvider$General
app_1  |    org.glassfish.jersey.jaxb.internal.XmlCollectionJaxbProvider$App
app_1  |    org.glassfish.jersey.message.internal.FormMultivaluedMapProvider
app_1  |    org.glassfish.jersey.jaxb.internal.XmlJaxbElementProvider$General
app_1  |    org.glassfish.jersey.jaxb.internal.XmlRootElementJaxbProvider$General
app_1  |    org.glassfish.jersey.message.internal.SourceProvider$SaxSourceReader
app_1  |    org.glassfish.jersey.jaxb.internal.DocumentProvider
app_1  |    com.fasterxml.jackson.jaxrs.json.JacksonJsonProvider
app_1  |    org.glassfish.jersey.message.internal.InputStreamProvider
app_1  |    com.fasterxml.jackson.jaxrs.xml.JacksonXMLProvider
app_1  |    org.glassfish.jersey.message.internal.BasicTypesMessageProvider
app_1  |    org.glassfish.jersey.jaxb.internal.XmlRootElementJaxbProvider$App
app_1  |    org.glassfish.jersey.jaxb.internal.XmlRootObjectJaxbProvider$App
app_1  |    org.glassfish.jersey.jaxb.internal.XmlRootObjectJaxbProvider$Text
app_1  |    org.glassfish.jersey.message.internal.ReaderProvider
app_1  |    org.glassfish.jersey.message.internal.SourceProvider$StreamSourceReader
app_1  |    org.glassfish.jersey.message.internal.StringMessageProvider
app_1  |    org.glassfish.jersey.message.internal.FileProvider
app_1  |    org.glassfish.jersey.jaxb.internal.XmlJaxbElementProvider$App
app_1  |    org.glassfish.jersey.jaxb.internal.XmlRootElementJaxbProvider$Text
app_1  |    org.glassfish.jersey.message.internal.SourceProvider$DomSourceReader
app_1  |    org.glassfish.jersey.message.internal.RenderedImageProvider
app_1  |    org.glassfish.jersey.message.internal.DataSourceProvider
app_1  | Message Body Writers:
app_1  |    org.glassfish.jersey.message.internal.SourceProvider$SourceWriter
app_1  |    org.glassfish.jersey.message.internal.FormProvider
app_1  |    org.glassfish.jersey.jaxb.internal.XmlJaxbElementProvider$Text
app_1  |    org.glassfish.jersey.jaxb.internal.XmlCollectionJaxbProvider$General
app_1  |    org.glassfish.jersey.jaxb.internal.XmlCollectionJaxbProvider$Text
app_1  |    org.glassfish.jersey.message.internal.ByteArrayProvider
app_1  |    org.glassfish.jersey.jaxb.internal.XmlCollectionJaxbProvider$App
app_1  |    org.glassfish.jersey.message.internal.FormMultivaluedMapProvider
app_1  |    org.glassfish.jersey.message.internal.StreamingOutputProvider
app_1  |    org.glassfish.jersey.jaxb.internal.XmlJaxbElementProvider$General
app_1  |    org.glassfish.jersey.jaxb.internal.XmlRootElementJaxbProvider$General
app_1  |    org.glassfish.jersey.jaxb.internal.DocumentProvider
app_1  |    com.fasterxml.jackson.jaxrs.json.JacksonJsonProvider
app_1  |    org.glassfish.jersey.message.internal.InputStreamProvider
app_1  |    com.fasterxml.jackson.jaxrs.xml.JacksonXMLProvider
app_1  |    org.glassfish.jersey.message.internal.BasicTypesMessageProvider
app_1  |    org.glassfish.jersey.jaxb.internal.XmlRootElementJaxbProvider$App
app_1  |    org.glassfish.jersey.server.ChunkedResponseWriter
app_1  |    org.glassfish.jersey.message.internal.ReaderProvider
app_1  |    org.glassfish.jersey.message.internal.StringMessageProvider
app_1  |    org.glassfish.jersey.message.internal.FileProvider
app_1  |    org.glassfish.jersey.media.multipart.internal.MultiPartWriter
app_1  |    org.glassfish.jersey.jaxb.internal.XmlJaxbElementProvider$App
app_1  |    org.glassfish.jersey.jaxb.internal.XmlRootElementJaxbProvider$Text
app_1  |    org.glassfish.jersey.message.internal.RenderedImageProvider
app_1  |    org.glassfish.jersey.message.internal.DataSourceProvider
app_1  |
app_1  | 14:09:26.698 INFO (NetworkListener.java:750) Started listener bound to [0.0.0.0:8080]
app_1  | 14:09:26.707 INFO (NetworkListener.java:750) Started listener bound to [0.0.0.0:8081]
app_1  | 14:09:26.710 INFO (HttpServer.java:300) [HttpServer] Started.
app_1  | 14:09:26.711 INFO (GrizzlyServer.java:153) Grizzly server running.
app_1  | 14:30:46.254 WARN (Graph.java:918) graph contains no agencies (yet); API request times will be interpreted as GMT.
app_1  | 14:30:46.380 ERROR (OTPExceptionMapper.java:16) Unhandled exception
app_1  | org.opentripplanner.routing.error.TransitTimesException: null
app_1  |    at org.opentripplanner.routing.core.RoutingContext.check(RoutingContext.java:360) ~[otp.jar:1.1]
app_1  |    at org.opentripplanner.routing.core.RoutingRequest.setRoutingContext(RoutingRequest.java:1045) ~[otp.jar:1.1]
app_1  |    at org.opentripplanner.routing.core.RoutingRequest.setRoutingContext(RoutingRequest.java:1058) ~[otp.jar:1.1]
app_1  |    at org.opentripplanner.api.resource.SurfaceResource.createSurface(SurfaceResource.java:75) ~[otp.jar:1.1]
app_1  |    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_212]
app_1  |    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:1.8.0_212]
app_1  |    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_212]
app_1  |    at java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_212]
app_1  |    at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory$1.invoke(ResourceMethodInvocationHandlerFactory.java:81) ~[otp.jar:1.1]
app_1  |    at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:144) ~[otp.jar:1.1]
app_1  |    at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:161) ~[otp.jar:1.1]
app_1  |    at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:160) ~[otp.jar:1.1]
app_1  |    at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:99) ~[otp.jar:1.1]
app_1  |    at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:389) ~[otp.jar:1.1]
app_1  |    at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:347) ~[otp.jar:1.1]
app_1  |    at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:102) ~[otp.jar:1.1]
app_1  |    at org.glassfish.jersey.server.ServerRuntime$2.run(ServerRuntime.java:308) ~[otp.jar:1.1]
app_1  |    at org.glassfish.jersey.internal.Errors$1.call(Errors.java:271) [otp.jar:1.1]
app_1  |    at org.glassfish.jersey.internal.Errors$1.call(Errors.java:267) [otp.jar:1.1]
app_1  |    at org.glassfish.jersey.internal.Errors.process(Errors.java:315) [otp.jar:1.1]
app_1  |    at org.glassfish.jersey.internal.Errors.process(Errors.java:297) [otp.jar:1.1]
app_1  |    at org.glassfish.jersey.internal.Errors.process(Errors.java:267) [otp.jar:1.1]
app_1  |    at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:317) [otp.jar:1.1]
app_1  |    at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:291) [otp.jar:1.1]
app_1  |    at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:1140) [otp.jar:1.1]
app_1  |    at org.glassfish.jersey.grizzly2.httpserver.GrizzlyHttpContainer.service(GrizzlyHttpContainer.java:375) [otp.jar:1.1]
app_1  |    at org.glassfish.grizzly.http.server.HttpHandler$1.run(HttpHandler.java:224) [otp.jar:1.1]
app_1  |    at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:565) [otp.jar:1.1]
app_1  |    at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.run(AbstractThreadPool.java:545) [otp.jar:1.1]
app_1  |    at java.lang.Thread.run(Thread.java:748) [na:1.8.0_212]
app_1  | 15:09:15.503 INFO (DeferredAuthorityFactory.java:210) Disposing class org.geotools.referencing.factory.epsg.ThreadedHsqlEpsgFactory backing store
app_1  | 15:09:15.510 INFO (null:-1) Database closed
app_1  | 15:22:58.662 INFO (LuceneIndex.java:143) Starting background Lucene indexing.
app_1  | 15:23:16.111 INFO (LuceneIndex.java:91) Built Lucene index in 17447 msec

[zabuTNT]

I don't see GTFS zip file inside /opt/odh-mentor-otp ... are you sure that the graph was built with GTFS?

This is confirmed by the Logs, too. 14:30:46.254 WARN (Graph.java:918) graph contains no agencies (yet); API request times will be interpreted as GMT.

[alexlanz]

Can I delete all the existing files inside /opt/odh-mentor-otp and reexecute the build task? Or must I leave some files inside this folder?

With deleting the files I could check if the build task is executing correctly.

[zabuTNT]

@alexlanz you don't need to remove and re-download the maps, (and don't remove osm.url). You can manually add the GTFS files and re-run the build task only. The GTFS file (temporary) it's here: https://cloud.opendatahub.bz.it/index.php/s/MRsMxr8wkNW3Xpd

In alternative, at the link above, you can find a Graph already built , so you can download it (it's compressed in gz, you have to decompress it) and put it inside /opt/odh-mentor-otp/openmove folder. Then re-run otp.

[alexlanz]

Sorry, but I don't understand what I have to do. 😥 I only received a quick introduction with very little insights. Could you please examplain it to me in more detail?

The current explanation that I receivied is the following:

1. Create two server with the corresponding piplines that build/execute on time the build app and one time the web app. The build app should be hosted on the machine with lots of memory
2. Make a fileshare between those two machines (/opt/odh-mentor-otp)
3. Run the pipelines

That's it. Could you please provide me more details about what should be inside the folder /opt/odh-mentor-otp? Or is this folder at the beginning empty? Because I would try to start from the beginning to see if everything works. If something should be placed inside this folder, then please add it to the readme so that we don't forget it in the future. If the must be nothing, then I would like to delete everything and rerun the build job.

[zabuTNT]

ok @alexlanz . at the beginning /opt/odh-mentor-otp should never been empty (for now). It must contain at least the

1. osm.url
2. a valid GTFS zip files

then the "build machine" can download the maps based on osm.url. https://github.com/noi-techpark/odh-mentor-otp/blob/development/infrastructure/docker/docker-entrypoint.sh#L3

when the folder contains all the maps, the gtfs file and the tif file, you can perform the build of the graph: https://github.com/noi-techpark/odh-mentor-otp/blob/development/infrastructure/docker/docker-entrypoint.sh#L68

this will create a obj file inside /opt/odh-mentor-otp/openmove. At this point the the "build machine" can be power off and the "execute machine" have to relaunch OTP, so it will use the new graph. https://github.com/noi-techpark/odh-mentor-otp/blob/development/infrastructure/docker/docker-entrypoint.sh#L100

---

With the next tasks we will introduce:

1. GTFS automatic download
2. automatic osm.url generation based on GTFS downloaded

3. automatic re-launch of otp inside execute machine when a new generated graph is detected

   ---

It's more clear now?

[ghost]

I add a clarification. The "execute machine" require only one file: /opt/odh-mentor-otp/openmove/Graph.obj

every time the gtfs data changes, it must be regenerated by the "build machine", to rebuild the Graph.obj file it always need the files: /opt/odh-mentor-otp/*_ExportGTFS.zip (newest gtfs data) /opt/odh-mentor-otp/srtm_39_03.zip (altimetric data) /opt/odh-mentor-otp/*.osm (openstreetmap data)

the download operation of the last two may not be inserted in the normal gtfs update pipeline: altimetric data: they will never change in the future, never openstreetmap data: changes very rarely, probably every 12/24 months

for this reason the osm data download mechanism is currently not integrated in docker(for now). Instead is important that the gtfs download is automatic

[alexlanz]

I tried now to set up the folder and to follow the "Usage" section of the readme: https://github.com/noi-techpark/odh-mentor-otp/blob/development/README.md#usage

But the command wget http://example.source.gtfs.com/200804_ExportGTFS.zip fails with the message:

wget: unable to resolve host address 'example.source.gtfs.com'

[rcavaliere]

@alexlanz the link is not correct, it's clearly an example. The GTFS export is available here: https://cloud.opendatahub.bz.it/index.php/s/5QfFk7rw47dzWGc

@stemove @zabuTNT maybe it's better to improve a little bit the documentation here

[alexlanz]

Ok, now a made a retry of the whole setup including setting up the directory /opt/odh-mentor-otp/openmove/ like described in the readme. Here are the logs from the server:

root@ip-172-31-11-34:/var/docker/odh-mentor-otp-run/current# ls -la /opt/odh-mentor-otp/
total 3412600
drwxr-xr-x 4 root root      6144 Sep 30 08:46 .
drwxr-xr-x 5 root root      4096 Sep 24 08:49 ..
-rw-r--r-- 1 root root 132596643 Sep 30 08:32 10_397045932724035_45_94607517243942_10_91385821636202_46_30580331792924.osm
-rw-r--r-- 1 root root 137624521 Sep 30 08:32 10_397045932724035_46_30580331792924_10_91385821636202_46_66553146341906.osm
-rw-r--r-- 1 root root 102802873 Sep 30 08:32 10_397045932724035_46_66553146341906_10_91385821636202_47_025259608908875.osm
-rw-r--r-- 1 root root 186360014 Sep 30 08:32 10_397045932724035_47_025259608908875_10_91385821636202_47_38498775439869.osm
-rw-r--r-- 1 root root 298087481 Sep 30 08:33 10_91385821636202_45_94607517243942_11_430670500000003_46_30580331792924.osm
-rw-r--r-- 1 root root 292741005 Sep 30 08:33 10_91385821636202_46_30580331792924_11_430670500000003_46_66553146341906.osm
-rw-r--r-- 1 root root 186957735 Sep 30 08:33 10_91385821636202_46_66553146341906_11_430670500000003_47_025259608908875.osm
-rw-r--r-- 1 root root       711 Sep 30 08:34 10_91385821636202_47_025259608908875_11_430670500000003_47_38498775439869.osm
-rw-r--r-- 1 root root 287003995 Sep 30 08:34 11_430670500000003_45_94607517243942_11_947482783637987_46_30580331792924.osm
-rw-r--r-- 1 root root 244948968 Sep 30 08:34 11_430670500000003_46_30580331792924_11_947482783637987_46_66553146341906.osm
-rw-r--r-- 1 root root 230544713 Sep 30 08:35 11_430670500000003_46_66553146341906_11_947482783637987_47_025259608908875.osm
-rw-r--r-- 1 root root 273387283 Sep 30 08:35 11_430670500000003_47_025259608908875_11_947482783637987_47_38498775439869.osm
-rw-r--r-- 1 root root 145861797 Sep 30 08:35 11_947482783637987_46_30580331792924_12_464295067275971_46_66553146341906.osm
-rw-r--r-- 1 root root 149304286 Sep 30 08:36 11_947482783637987_46_66553146341906_12_464295067275971_47_025259608908875.osm
-rw-r--r-- 1 root root 152826734 Sep 30 08:36 11_947482783637987_47_025259608908875_12_464295067275971_47_38498775439869.osm
-rw-r--r-- 1 root root 196274690 Sep 30 08:36 12_464295067275971_46_30580331792924_12_981107350913955_46_66553146341906.osm
-rw-r--r-- 1 root root 212574392 Sep 30 08:37 12_464295067275971_46_66553146341906_12_981107350913955_47_025259608908875.osm
drwxr-xr-x 2 root root      6144 Sep 30 08:29 200804_ExportGTFS
-rw-r--r-- 1 root root 103359175 Sep 30 08:31 9_880233649086051_46_30580331792924_10_397045932724035_46_66553146341906.osm
-rw-r--r-- 1 root root  89085026 Sep 30 08:32 9_880233649086051_46_66553146341906_10_397045932724035_47_025259608908875.osm
drwxr-xr-x 2 root root      6144 Sep 30 08:46 openmove
-rw-r--r-- 1 root root      2109 Sep 30 08:28 osm.url
-rw-r--r-- 1 root root  72096675 Sep 30 08:31 srtm_39_03.tif
root@ip-172-31-11-34:/var/docker/odh-mentor-otp-run/current# docker-compose logs
Attaching to odh-mentor-otp-run_app_1
app_1  | 09:02:09.348 INFO (OTPServer.java:39) Wiring up and configuring server.
app_1  | 09:02:09.354 INFO (GraphScanner.java:64) Attempting to automatically register routerIds [openmove]
app_1  | 09:02:09.355 INFO (GraphScanner.java:65) Graph files will be sought in paths relative to /data
app_1  | 09:02:09.356 INFO (GraphService.java:176) Registering new router 'openmove'
app_1  | 09:02:09.369 INFO (InputStreamGraphSource.java:181) Loading graph...
app_1  | 09:02:10.051 INFO (Graph.java:746) Graph version: MavenVersion(1, 4, 0, , b272f14007c97d769216e9ebab65baad7410cdf5)
app_1  | 09:02:10.051 INFO (Graph.java:747) OTP version:   MavenVersion(1, 4, 0, , b272f14007c97d769216e9ebab65baad7410cdf5)
app_1  | 09:02:10.051 INFO (Graph.java:764) This graph was built with the currently running version and commit of OTP.
app_1  | 09:02:16.268 INFO (Graph.java:731) Main graph read. |V|=459845 |E|=1114708
app_1  | 09:02:25.992 INFO (GraphIndex.java:127) Indexing graph...
app_1  | 09:02:27.071 INFO (GraphIndex.java:595) Clustering stops by geographic proximity and name...
app_1  | 09:02:27.072 INFO (GraphIndex.java:210) Creating a spatial index for stop clusters.
app_1  | 09:02:27.117 INFO (GraphIndex.java:192) Initializing areas....
app_1  | 09:02:27.118 INFO (GraphIndex.java:199) Done indexing graph.
app_1  | 09:02:27.334 INFO (Router.java:100) No default routing parameters were found in the router config JSON. Using built-in OTP defaults.
app_1  | 09:02:27.341 INFO (Router.java:127) Timeouts for router 'openmove': [5.0, 4.0, 2.0]
app_1  | 09:02:27.342 INFO (Router.java:134) Incoming requests will not be logged.
app_1  | 09:02:27.343 INFO (GraphUpdaterConfigurator.java:41) Using configurations: [main]
app_1  | 09:02:27.527 INFO (Router.java:172) Computed ellipsoid/geoid offset at (46.6467887, 11.435274249999999) as 50.6343278014937
app_1  | 09:02:27.532 INFO (GrizzlyServer.java:72) Starting OTP Grizzly server on ports 8080 (HTTP) and 8081 (HTTPS) of interface 0.0.0.0
app_1  | 09:02:27.532 INFO (GrizzlyServer.java:74) OTP server base path is /var/otp
app_1  | 09:02:27.578 INFO (GrizzlyServer.java:51) Java reports that this machine has 2 available processors.
app_1  | 09:02:27.579 INFO (GrizzlyServer.java:62) Maximum HTTP handler thread pool size will be 4 threads.
app_1  | 09:02:27.629 INFO (ServiceFinder.java:172) Running in a non-OSGi environment
app_1  | 09:02:27.893 INFO (ApplicationHandler.java:419) Initiating Jersey application, version Jersey: 2.18 2015-06-05 02:28:21...
app_1  | 09:02:28.699 INFO (ExtendedLogger.java:345) Selected ExecutorServiceProvider implementation [org.glassfish.jersey.server.internal.process.ServerProcessingBinder$DefaultManagedAsyncExecutorProvider] to be used for injection of executor qualified by [org.glassfish.jersey.server.ManagedAsyncExecutor] annotation.
app_1  | 09:02:28.703 INFO (ExtendedLogger.java:345) Selected ScheduledExecutorServiceProvider implementation [org.glassfish.jersey.server.internal.process.ServerProcessingBinder$DefaultBackgroundSchedulerProvider] to be used for injection of scheduler qualified by [org.glassfish.jersey.server.BackgroundScheduler] annotation.
app_1  | 09:02:28.908 WARN (PropertiesHelper.java:330) There is no way how to transform value "true" [java.lang.Boolean] to type [java.lang.String].
app_1  | 09:02:28.913 INFO (ApplicationHandler.java:683) Jersey application initialized.
app_1  | Root Resource Classes:
app_1  |   org.opentripplanner.api.resource.SIsochrone
app_1  |   org.opentripplanner.api.resource.SimpleIsochrone
app_1  |   org.opentripplanner.api.resource.PlannerResource
app_1  |   org.opentripplanner.api.resource.TimeGridWs
app_1  |   org.opentripplanner.index.GeocoderResource
app_1  |   org.opentripplanner.api.resource.ScriptResource
app_1  |   org.opentripplanner.api.resource.ServerInfo
app_1  |   org.opentripplanner.api.resource.SurfaceResource
app_1  |   org.opentripplanner.api.resource.ExternalGeocoderResource
app_1  |   org.opentripplanner.api.resource.AlertPatcher
app_1  |   org.opentripplanner.api.resource.RepeatedRaptorTestResource
app_1  |   org.opentripplanner.api.resource.GraphInspectorTileResource
app_1  |   org.opentripplanner.api.resource.UpdaterStatusResource
app_1  |   org.opentripplanner.api.resource.Routers
app_1  |   org.opentripplanner.api.resource.LegendResource
app_1  |   org.opentripplanner.api.resource.ProfileResource
app_1  |   org.opentripplanner.api.resource.PointSetResource
app_1  |   org.opentripplanner.api.resource.LIsochrone
app_1  |   org.opentripplanner.api.resource.ScenarioResource
app_1  |   org.opentripplanner.index.IndexAPI
app_1  |   org.opentripplanner.api.resource.BikeRental
app_1  |   org.opentripplanner.api.resource.TileService
app_1  | Global Request Filters:
app_1  |    org.opentripplanner.standalone.AuthFilter
app_1  |    org.opentripplanner.standalone.CorsFilter
app_1  | Global Response Filters:
app_1  |    org.opentripplanner.standalone.CorsFilter
app_1  | Global Reader Interceptors:
app_1  |    org.glassfish.jersey.server.internal.MappableExceptionWrapperInterceptor
app_1  | Global Writer Interceptors:
app_1  |    org.glassfish.jersey.server.internal.MappableExceptionWrapperInterceptor
app_1  |    org.glassfish.jersey.server.internal.JsonWithPaddingInterceptor
app_1  | Dynamic Features:
app_1  |    org.glassfish.jersey.server.filter.RolesAllowedDynamicFeature@3a230001
app_1  | Message Body Readers:
app_1  |    org.glassfish.jersey.jaxb.internal.XmlRootElementJaxbProvider$Text
app_1  |    org.glassfish.jersey.message.internal.ByteArrayProvider
app_1  |    org.glassfish.jersey.jaxb.internal.DocumentProvider
app_1  |    org.glassfish.jersey.message.internal.RenderedImageProvider
app_1  |    org.glassfish.jersey.message.internal.ReaderProvider
app_1  |    org.glassfish.jersey.jaxb.internal.XmlRootObjectJaxbProvider$App
app_1  |    org.glassfish.jersey.message.internal.SourceProvider$SaxSourceReader
app_1  |    org.glassfish.jersey.jaxb.internal.XmlRootObjectJaxbProvider$General
app_1  |    org.glassfish.jersey.message.internal.InputStreamProvider
app_1  |    org.glassfish.jersey.message.internal.BasicTypesMessageProvider
app_1  |    org.glassfish.jersey.jaxb.internal.XmlRootElementJaxbProvider$General
app_1  |    com.fasterxml.jackson.jaxrs.json.JacksonJsonProvider
app_1  |    org.glassfish.jersey.jaxb.internal.XmlJaxbElementProvider$Text
app_1  |    org.glassfish.jersey.message.internal.StringMessageProvider
app_1  |    org.glassfish.jersey.jaxb.internal.XmlRootElementJaxbProvider$App
app_1  |    org.glassfish.jersey.message.internal.FormMultivaluedMapProvider
app_1  |    org.glassfish.jersey.message.internal.DataSourceProvider
app_1  |    org.glassfish.jersey.jaxb.internal.XmlRootObjectJaxbProvider$Text
app_1  |    org.glassfish.jersey.media.multipart.internal.MultiPartReaderServerSide
app_1  |    org.glassfish.jersey.message.internal.FileProvider
app_1  |    org.glassfish.jersey.jaxb.internal.XmlCollectionJaxbProvider$General
app_1  |    org.glassfish.jersey.message.internal.FormProvider
app_1  |    org.glassfish.jersey.jaxb.internal.XmlJaxbElementProvider$App
app_1  |    org.glassfish.jersey.jaxb.internal.XmlJaxbElementProvider$General
app_1  |    com.fasterxml.jackson.jaxrs.xml.JacksonXMLProvider
app_1  |    org.glassfish.jersey.message.internal.SourceProvider$StreamSourceReader
app_1  |    org.glassfish.jersey.jaxb.internal.XmlCollectionJaxbProvider$App
app_1  |    org.glassfish.jersey.jaxb.internal.XmlCollectionJaxbProvider$Text
app_1  |    org.glassfish.jersey.message.internal.SourceProvider$DomSourceReader
app_1  | Message Body Writers:
app_1  |    org.glassfish.jersey.jaxb.internal.XmlRootElementJaxbProvider$Text
app_1  |    org.glassfish.jersey.message.internal.ByteArrayProvider
app_1  |    org.glassfish.jersey.jaxb.internal.DocumentProvider
app_1  |    org.glassfish.jersey.message.internal.RenderedImageProvider
app_1  |    org.glassfish.jersey.message.internal.ReaderProvider
app_1  |    org.glassfish.jersey.message.internal.InputStreamProvider
app_1  |    org.glassfish.jersey.message.internal.BasicTypesMessageProvider
app_1  |    org.glassfish.jersey.jaxb.internal.XmlRootElementJaxbProvider$General
app_1  |    com.fasterxml.jackson.jaxrs.json.JacksonJsonProvider
app_1  |    org.glassfish.jersey.jaxb.internal.XmlJaxbElementProvider$Text
app_1  |    org.glassfish.jersey.message.internal.StringMessageProvider
app_1  |    org.glassfish.jersey.jaxb.internal.XmlRootElementJaxbProvider$App
app_1  |    org.glassfish.jersey.message.internal.FormMultivaluedMapProvider
app_1  |    org.glassfish.jersey.message.internal.SourceProvider$SourceWriter
app_1  |    org.glassfish.jersey.server.ChunkedResponseWriter
app_1  |    org.glassfish.jersey.message.internal.DataSourceProvider
app_1  |    org.glassfish.jersey.message.internal.FileProvider
app_1  |    org.glassfish.jersey.message.internal.StreamingOutputProvider
app_1  |    org.glassfish.jersey.jaxb.internal.XmlCollectionJaxbProvider$General
app_1  |    org.glassfish.jersey.media.multipart.internal.MultiPartWriter
app_1  |    org.glassfish.jersey.message.internal.FormProvider
app_1  |    org.glassfish.jersey.jaxb.internal.XmlJaxbElementProvider$App
app_1  |    org.glassfish.jersey.jaxb.internal.XmlJaxbElementProvider$General
app_1  |    com.fasterxml.jackson.jaxrs.xml.JacksonXMLProvider
app_1  |    org.glassfish.jersey.jaxb.internal.XmlCollectionJaxbProvider$App
app_1  |    org.glassfish.jersey.jaxb.internal.XmlCollectionJaxbProvider$Text
app_1  |
app_1  | 09:02:32.119 INFO (NetworkListener.java:750) Started listener bound to [0.0.0.0:8080]
app_1  | 09:02:32.122 INFO (NetworkListener.java:750) Started listener bound to [0.0.0.0:8081]
app_1  | 09:02:32.125 INFO (HttpServer.java:300) [HttpServer] Started.
app_1  | 09:02:32.125 INFO (GrizzlyServer.java:153) Grizzly server running.

Let me know what else I can do or if I made something wrong.

[zabuTNT]

Hi @alexlanz the GTFS have not to be unzipped (or at least the zip file have not to be deleted), I see that there is a folder:

drwxr-xr-x 2 root root 6144 Sep 30 08:29 200804_ExportGTFS

The graph builder don't use the subfolders, but only the zip/osm or tif files

[ghost]

@alexlanz the link is not correct, it's clearly an example. The GTFS export is available here: https://cloud.opendatahub.bz.it/index.php/s/5QfFk7rw47dzWGc

@stemove @zabuTNT maybe it's better to improve a little bit the documentation here

I'm working to rewrite the README more detailed in the boilerplate format I send you a PR today

[alexlanz]

Ok, thank you. I will wait for the readme and then try to find the missing step.

[ghost]

@alexlanz we are working on how to simplify the downloading osm data here: https://github.com/openmove/odh-mentor-otp/issues/3 at the moment it requires some tests for having this, and you can use the pregenerated osm.url file: https://github.com/openmove/odh-mentor-otp/blob/development/osm.url (I'm added this in the last PR)

another thing is to automate the automatic reload of the "running machine" service: https://otp-graph.opendatahub.testingmachine.eu/ when the graph is built: https://github.com/openmove/odh-mentor-otp/issues/17 for the moment the online machine must be restarted manually at each construction of the graph

[ghost]

hi @alexlanz

I'm just merged in the same PR the features I mentioned above

you will notice now that the deployment is much easier because the data download has been fully automated in the build container.

compared to before, only an environment variable has been introduced in which the name of the gtfs zip file must be specified: GTFS_FILE https://github.com/openmove/odh-mentor-otp/tree/development#docker-environment

for everything I have completely updated the README