otp-middleware

The otp-middleware application proxies requests from OpenTripPlanner UI to API, enhancing OpenTripPlanner (OTP) with user storage, real-time trip monitoring, and more!

Background

OTP provides multi-modal journey planning, combining transit, biking, walking, and various car options (e.g., Park & Ride) into complete passenger itineraries.

Transportation agencies might want to extend the functionality of their trip planner to improve the overall user experience for their OTP implementation. otp-middleware is a potential way for agencies to fill in those gaps. It provides:

• user management and storage,
• trip monitoring,
• and other features (TODO).

Deployment

The easiest way to deploy otp-middleware is to use the included Docker Compose configuration.

There are three compose configs. All build upon the "standard" config. The dev config exposes the Mongo instance, and saves the database data outside of the container. The path the database data is saved to is by default set to the directory homebrew installs Mongo into. This can be changed to any Mongo installation directory. The e2e configuration also exposes the Mongo instance, and pre-seeds the database using the files in the configurations/e2e-mongo-seed directory. Please note that the seed files do not work out of the box -- valid auth0 users are needed, and their information needs to be pasted into the seed json files. Fields which require input are clearly marked.

The configurations/default/docker.env.yml contains a configuration file that makes it easy to get started. However, there are fields which need to be manually set. These fields are clearly marked within the file.

Running otp-middleware via Docker Compose

docker compose -f docker-compose.yml up

Running otp-middleware in dev mode via Docker Compose

docker compose -f docker-compose.yml -f docker-compose.dev.yml up

Running otp-middleware in e2e mode via Docker Compose

Note: you will need to populate env.docker.yml, admin-e2euser.json, api-e2euser.json and aws-credentials

docker compose -f docker-compose.yml -f docker-compose.e2e.yml up

OTP Middleware is also suitable for continuous deployment implementations.

Development

To run otp-middleware in a local, development environment:

# Clone the repo.
git clone https://github.com/ibi-group/otp-middleware.git
cd otp-middleware
# Copy the config template, then make any changes to the file to 
# match your local configuration (e.g., database name).
cp configurations/default/env.yml.tmp configurations/default/env.yml
# At this point, you can either package the source code into an 
# executable jar, or import and run the code in an interactive 
# development environment. 
mvn package
java -jar target/otp-middleware.jar configurations/default/env.yml

It's also possible to run otp-middleware via Docker (see Deployment above)

Configuration

Auth0

TODO: Add Auth0 setup instructions.

Auth0 Scope

The requesting user type, which determines the level of authorization, is based on the scope provided as part of a user's bearer token. The bearer token 'scope' claim must contain one of otp-user, api-user or admin-user for the user to be correctly matched to a user held in the database.

Auth0 Scope Rule

A rule must be added to the Auth0 tenant for the scope provided by third parties to be available for authorization within the OTP-middleware. This rule takes the scope value provided by the caller and adds it to the access token.

function (user, context, callback) {
    const req = context.request;
    // Retrieve scopes either from the parameters or body
    const requestedScopeString = (req.query && req.query.scope) || (req.body && req.body.scope);
    context.accessToken.scope = requestedScopeString;
    return callback(null, user, context);
}

OTP Server Proxy Setup

The follow parameters are used to interact with an OTP server.

Trip Actions

OTP-middleware supports triggering certain actions when someone activates live tracking of a monitored trip and reaches a location or is about to enter a path. Actions include location-sensitive API calls to notify various services. In the context of live trip tracking, actions may include notifying transit vehicle operators or triggering traffic signals.

Bus Notify Actions

Bus notifier actions are defined in the optional file bus-notifier-actions.yml in the same configuration folder as env.yml. The file contains a list of actions defined by an agency ID and a fully-qualified trigger class:

- agencyId: id1
  trigger: com.example.package.MyTriggerClass

Known trigger classes below are in package org.opentripplanner.middleware.triptracker.interactions.busnotifiers and implement its BusOperatorInteraction interface:

Monitored Components

This application allows you to monitor various system components (e.g., OTP API, OTP UI, and Data Tools) that work together to provide a trip planning service. Each of these should be defined in the config file in the list of MONITORED_COMPONENTS with the following properties:

Bugsnag

Bugsnag is used to report error events that occur within the otp-middleware application or OpenTripPlanner components that it is monitoring.

Bugsnag Configuration Parameters

These values can be used as defined here (where applicable), or commented out, so the default values are used. Parameters that don't have default values (N/A) can be obtained by following the steps in the next section.

Bugsnag Setup

Where default parameters cannot be used, these steps describe how to obtain each compulsory parameter.

BUGSNAG_API_KEY

A bugsnag API key is a key that is unique to an individual Bugsnag user. This key can be obtained by logging into Bugsnag (https://app.bugsnag.com), clicking on settings (top right-hand corner) then My account settings. From here select Personal auth tokens and then Generate new token.

BUGSNAG_PROJECT_NOTIFIER_API_KEY

A Bugsnag project identifier key is unique to a Bugsnag project and allows errors to be saved against it. This key can be obtained by logging into Bugsnag (https://app.bugsnag.com), clicking on Projects (left side menu) and selecting the required project. Once selected, the notifier API key is presented.

Configure the Bugsnag Webhook

The Bugsnag webhook must be configured for your project, so that errors can be forwarded to the OTP Middleware and to anyone subscribing to get email notifications when errors occur.

To configure the Bugsnag Webhook: 1) Select the desired project in the Bugsnag console. 2) Under "Integrations and email" select "Data forwarding". 3) Under "Available integrations" select "Webhook". 4) Enter the URL you would like Bugsnag to push project errors to e.g. :/api/bugsnagwebhook.

More information on the Bugsnag webhook can be found here: https://docs.bugsnag.com/product/integrations/data-forwarding/webhook/

Whitelisting addresses from BUGSNAG_WEBHOOK_PERMITTED_IPS

In some restricted environments such as AWS Security Groups, you may need to whitelist the two Bugsnag IP addresses, so that Bugsnag can post error notifications to OTP Middleware and that the errors appear in the admin dashboard. Refer to your cloud service for whitelisting IP addresses.

Connected Data Platform

AWS S3 Policy configuration

An IAM access management S3 policy is required in order for an IAM user to write/delete objects on the Connected Data Platform S3 bucket.

The following permissions are required: 1) ListBucket 2) GetObject 3) DeleteObject 4) PutObject 5) PutObjectAcl

The following snippet is an example policy which can be used/modified to allow access to the CDP S3 bucket:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "VisualEditor0",
      "Effect": "Allow",
      "Action": [
        "s3:PutObject",
        "s3:GetObject",
        "s3:ListBucket",
        "s3:DeleteObject",
        "s3:PutObjectAcl"
      ],
      "Resource": [
        "arn:aws:s3:::cdp-bucket-name/*",
        "arn:aws:s3:::cdp-bucket-name"
      ]
    }
  ]
}

Testing

End-to-end (E2E)

In order to run E2E tests, specific configuration and environment variables must be used.

Please note the e2e Dockerfile is not used to run the otp-middleware e2e tests. These are instead used to instantiate a standard otp-middleware instance which can be used by other application's e2e tests.

Auth0

The standard Auth0 configuration can be used for the server settings. However, some additional settings must be applied in order for the server to get an oath token from Auth0 for creating authenticated requests. A private application should be created for use in Auth0 following these instructions: https://auth0.com/docs/flows/call-your-api-using-resource-owner-password-flow

One special note is that the default directory must be set at the Account/tenant level. Otherwise, authorization will not be successful. See this StackOverflow answer for more info: https://stackoverflow.com/a/43835563/915811

The special E2E client settings should be defined in env.yml:

Note: Just to reiterate, these are different from the server application settings and are only needed for E2E testing.

env.schema.json values