Investigate and setup Transitime to generate our own realtime predictions

[luqmaan]

Transitime is an open-source real-time transit information system. It combines readily available GPS data and GTFS data to provide real-time arrival information such as "next bus is in 5 minutes".

Transitime sounds amazing! Lets set it up and document our progress here.

If we're able to get Transitime working, we can feed the predictions into OneBusAway and get a OneBusAway server running for Austin.

http://www.transitime.org https://github.com/Transitime/core/wiki/ http://transitime.github.io/core/javadocs/

https://github.com/open-austin/project-ideas/issues/21

:bus::dash::dash::dash:

[luqmaan]

Can use this to generate Trip Updates feed:

https://github.com/Transitime/core/wiki/API#gtfs-realtime-commands

The tripUpdates command for outputting prediction information in GTFS-realtime format. Can use query string option format=human for human readable output. http://DOMAIN/api/v1/key/TEST_KEY/agency/sfmta/command/gtfs-rt/tripUpdates?OPTIONS

And I guess this is where we link to the vehicle position feed for gtfs rt?

https://github.com/Transitime/core/blob/08561cbfc57ddf39bfd24a8590dc65dda2152b0d/transitime/src/test/resources/testConfig.xml

[walkeriniraq]

Stream of consciousness from getting this running:

Downloaded source and packaged with "mvn install -DskipTests"

SQL schema in transitime-core/transitime/target/classes/ddl_*

java -jar processGTFSFile.jar -c db.properties -gtfsDirectoryName ~/Downloads/transit/

db.properties:
transitime.db.dbName=transitime
transitime.db.dbType=postgresql
transitime.db.dbUserName=nathan
transitime.db.dbPassword=*******
transitime.hibernate.configFile=postgres_hibernate.cfg.xml

postgres_hibernate.cfg.xml copied from target/classes folder

Run with:
java -Dtransitime.hibernate.configFile=postgres_hibernate.cfg.xml -Dtransitime.avl.gtfsRealtimeFeedURI="https://data.texas.gov/download/i5qp-g5fd/application/octet-stream" -Dtransitime.modules.optionalModulesList=org.transitime.avl.GtfsRealtimeModule -Dtransitime.core.agencyId=transitime -jar core.jar

CURRENT STATUS: failing to correctly parse data from CapMetro.

Failure from log:
ERROR thread=avlClient-1 [o.t.a.AvlClient:82] Throwing away avlReport AvlReport [vehicleId=9310, time=01-17-1970 10:19:18.198 CST, location=[30.48167, -97.78945], speed=7.7m/s, heading=NaN, source=GTFS-rt, assignmentId=983, assignmentType=ROUTE_ID] because Time of 01-17-1970 10:19:18 CST is more than 10 years old. 

message example:
entity {
  id: "159"
  vehicle {
    trip {
      trip_id: "1491703"
      start_time: "20:52"
      start_date: "20150901"
      route_id: "983"
    }
    position {
      latitude: 30.40891
      longitude: -97.74563
      speed: 37.52
    }
    timestamp: 1441158775
    vehicle {
      id: "9103"
    }
  }
}

[walkeriniraq]

Added a PR for fixing a bug in transitime GTFS-RT to AVL timestamps here: https://github.com/Transitime/core/pull/16

[walkeriniraq]

I have the webapps up and running in tomcat. Fairly easy to deploy.

1) Download tomcat and extract. 2) Extract api.war and web.war inside of the tomcat/webapps folder. 3) Change web/WEB-INF/web.xml to point to correct properties file with database info 4) Change api/WEB-INF/web.xml to point to correct properties file with database info 5) Use transitime.db.webstructs.WebAgency to add record to database. 6) Start tomcat

[luqmaan]

:fireworks: :fireworks: :fireworks:

Nice work man!

Now we need to get it running on a server.

[luqmaan]

Should we meet up sometime next week to get started on that?

I think it would be worthwhile to get the stuff running in a Docker container.

http://azure.microsoft.com/en-us/marketplace/partners/canonical/ubuntuserver1504/ http://azure.microsoft.com/en-us/marketplace/partners/canonicalandmsopentech/dockeronubuntuserver1404lts/

[walkeriniraq]

Let's make sure I can get predictions working first. On Sep 7, 2015 12:31 PM, "Luqmaan Dawoodjee" notifications@github.com wrote:

Should we meet up sometime next week to get started on that?

I think it would be worthwhile to get the stuff running in a Docker container.

http://azure.microsoft.com/en-us/marketplace/partners/canonical/ubuntuserver1504/

http://azure.microsoft.com/en-us/marketplace/partners/canonicalandmsopentech/dockeronubuntuserver1404lts/

— Reply to this email directly or view it on GitHub https://github.com/luqmaan/Instabus/issues/210#issuecomment-138346320.

[luqmaan]

We've made some good progress.

https://github.com/walkeriniraq/transittime-core/tree/cap-metro/transitime https://github.com/walkeriniraq/transitime-db https://github.com/walkeriniraq/transitime-db/blob/luq/README.md

[luqmaan]

It lives!

http://transitime-host.cloudapp.net/api/v1/key/f18a8240/agency/cap-metro/command/predictions?rs=201%7C5315

http://transitime-host.cloudapp.net/api/v1/key/f18a8240/agency/cap-metro/command/predictions?rs=801%7C5864

http://transitime-host.cloudapp.net/api/v1/key/f18a8240/agency/cap-metro/command/gtfs-rt/tripUpdates?format=human

[ndarilek]

Not sure if this comment is off-topic for this issue. I found it because I'm trying to get a OneBusAway instance running for Austin. Was this issue concerned with that as well, or only for creating a predictions feed that could be input to OBA?

I ask because I'm having a rough time getting an OBA instance running. Looks like CapMetro's GTFS data has a bunch of errors (~5000 according to the GTFS feed validator) and I'm wondering if/how you may have worked around that for this and other projects.

Thanks.

[luqmaan]

Hey Nolan,

We're also trying to get a OBA instance running for Austin. We ended up finding Transitime because we need a Trip Updates feed.

Lets work on setting up OBA/Transitime together.

Are you free tomorrow, Tuesday Dec 1 at 6:30pm? We're gonna be working on this project at the Open Austin Civic Hack Night. http://www.meetup.com/Open-Government-Civic-Technology-Meetup-by-Open-Austin/events/226908119/

[ndarilek]

Hmm, not free tomorrow unfortunately, but I've wanted to see a project like this for quite some time so perhaps next time.

I did, however, eventually manage to get an OBA instance running using the trip feed posted above and the script to fix GTFS data from one of the Transitime repositories. The setup runs in a Docker container inside Vagrant, and I'm currently building a GraphQL API on top of OBA's. The latter isn't anywhere near complete, but I can extract and push the Vagrant/Docker setup somewhere if it would save you time tomorrow. I do get lots of errors in the OBA logs concerning real-time polling--I'm wondering if there's a mismatch between the fixed GTFS data and the GTFS real-time feeds.

The setup isn't quite automatic, but it does work with a bit of manual prodding.

[luqmaan]

if there's a mismatch between the fixed GTFS data and the GTFS real-time feeds

Very likely.

building a GraphQL API on top of OBA's

Really cool. We're building a React app to around the OBA API. For now we're just using Tampa's instance since we don't have a useful one for Austin. https://github.com/open-austin/instabus

[luqmaan]

@ndarilek How far have you gotten with your GraphQL API?

Also, come join us on Slack. http://slack.open-austin.org. There's a #instabus channel and a #transitime channel/

[ndarilek]

I've wrapped the OBA route/, arrivals-and-departures-for-stop/ and stops-for-location/ API calls. The following query works through my custom GraphQL server against the OBA REST API:

{
   stopsForLocation(lat: 30.267153, lon: -97.7430608) {
     name
     arrivalsAndDepartures {
       route {
         shortName
         longName
         description
       }
     }
   }
}

and delivers, in one request/response cycle, data about the stops and next departures for the above position in a nice JSON payload. I'll next be adding other entrypoints so you can query by stop, route, trip, etc. I don't know a whole lot about GraphQL/Relay, so this is very much an excuse for me to learn and play around.

I'd love to join you all on Slack, but I'm blind and it is pretty much inaccessible to me as a screen reader user. The IRC interface also doesn't work on Thunderbird or Instantbird, and I haven't set up a shell-based IRC environment in years. Even so, I vastly prefer GUIs to shells these days. I'd love to see this group transition to a more open solution like rocket.chat, perhaps running in Sandstorm for easy administration, but I'm not really involved beyond being tangentially aware of it. Then again, an accessible chat system might facilitate me being more involved, so... :)

[luqmaan]

Its unfortunate that Slack app doesn't work with screen readers. I had no idea. Do screen readers work on the Slack webapp https://open-austin.slack.com/messages?

On our side, we've taken the steps to setup the IRC/XMPP gateway for Slack. https://slack.zendesk.com/hc/en-us/articles/201727913-Connecting-to-Slack-over-IRC-and-XMPP

[ndarilek]

I hadn't tried the app at all, but I'm almost certain its accessibility is worse than the web app.

When I last looked, it seemed like the IRC gateway was enabled but the XMPP gateway wasn't. I'll check again.

[luqmaan]

When I last looked, it seemed like the IRC gateway was enabled but the XMPP gateway wasn't. I'll check again.

Fixed now. https://open-austin.slack.com/account/gateways.

[luqmaan]

@ndarilek CapMetro has updated their GTFS zip and its working great. Here's the results of the feed validator: http://luqmaan.com/capmetro-gtfs-health/

I also got OBA running locally using the steps here: https://github.com/luqmaan/capmetrobeta/blob/master/README.md

:fireworks:

CapMetro has a working Trip Updates JSON dump (https://data.texas.gov/Transportation/Trip-Updates-JSON-File/mqtr-wwpy). They are still working on getting the Trip Updates PB feed. Once that is ready, I think OBA can serve realtime info.