Implement OneBusAway Glassware for Google Glass

[barbeau]

Having real-time transit information available for favorite/bookmarked stops from OneBusAway via Google Glass seems pretty compelling. Viewing estimated arrival transit info seems to fit the Glass model of quickly checking information well, with typical interactions that last a very short amount of time (e.g., 15 seconds). This ticket is intended to capture what we know about OBA and Glass, and track any progress towards implementation.

It seems that OBA Glassware could be implemented fairly easily using the Google Mirror API. In this model, the OBA Glassware is actually a web application that pushes information to Glass devices, and receives callbacks from user actions, via Google's Mirror API services. The Mirror API then takes care of syncing information with the user's Glass, and pushing callbacks triggered by user actions to the OBA Glassware service.

Installation process for Glassware

Typical "installation" process for Glassware is:

1. User visits website for Glassware via desktop or mobile app browser.
2. User opts-into using the Glassware, including various settings for how frequently information is updated and the number of notification, which results in the Glassware being "installed" on the user's Glass. For OBA, in the first iteration I imagine that the user would choose which bookmarked stops from their web browser they want to make available in the OBA Glassware (and potentially later, starred stops from mobile apps).
3. User interacts with cards on Glass to quickly view relevant real-time information. For OBA, I imagine this would involve seeing a list of favorite stops (including some logic to sort with the closest stop on top), and viewing the real-time arrival info.

   Implementation

Since Glassware is a webapp, it would be implemented in this Github project, with the UI components in the onebusaway-webapp project, and potentially some of the backend Mirror API operations implemented via the onebusaway-api-webapp.

We should probably wait to implement anything in the official project until #26 (Update to Google Maps API v3, including getting rid of Google Web Toolkit (GWT)) is resolved by Sound Transit, since the method of implementing a new webpage UI to control OBA Glassware settings will be different after GWT is removed. Also, we really need to get our hands on a real Glass device to test the app (and the actual Mirror API is currently only open to developers with the Google Glass Explorer Edition). But, we can get started with technical and UI design now.

OBA Glassware Design

Protocol

Based on my understanding of the Google Mirror API Usage Stories documentation, here is the proposed usage of the Mirror API to implement OBA Glassware:

1. OBA user visits onebusaway-webapp and subscribes to OBA Glassware by authenticating with OAuth 2.0 (note we're already using OAuth, as evidenced here). The user selects which (bookmarked) stops they want available via Glass. In addition to the usual glass.timeline scope, we also request glass.location scope access.
2. After OAuth2.0 completes, OBA Glassware subscribes to updates for the user's location.
3. OBA Glassware fetches the user's latest location from the location endpoint
4. OBA Glassware orders the selected user stops by proximity to the user, and creates a timeline card that lists the OBA stops and current estimated arrival times.
5. To keep this card on the left side of the clock screen, OBA Glassware pins the card.
6. As the user moves around, their last location is updated
7. Because OBA Glassware is subscribed to location updates, a notification is to to OBA Glassware by the Mirror API. This notification links to the latest item in the location collection.
8. Based on the notification, OBA Glassware service fetches the user's latest location from the location collection.
9. OBA Glassware re-orders the stops based on user proximity and updates the arrival time information.
10. OBA Glassware uses the patch method to update the body of the timeline item inserted in step 5.
11. The timeline item also includes a Refresh menu item that manually triggers a notification to the OBA Glassware service. Steps 8, 9, 10 would be performed.

Google Doc Drawing for above protocol image is here.

Glass UI Design

Since Glass is a completely different user experience than desktop or mobile, the UI design for the OBA cards seen by Glass users should be examined carefully, and tested in real-world situations with actual Glass devices. The recommended background color is black, since this will be transparent to the user. And since the background is transparent, high-contrast colors are recommended. I'm not sure how the default green/red/blue/black colors that we've used for ontime/early/late/no info will come across.

To get us started, here's a possible OBA card mockup for the estimated arrival times card, based on an existing transit example from the Google Mirror API Playground:

The "just now" text in the bottom right-hand corner is a default part of the card indicating when it was updated.

You can paste the below HTML into the Mirror API Playground page (just click on the second card template at the bottom that uses HTML, and then the "HTML" button before pasting into textbox), and it will render the above image that you can then edit and play with:

<article>
  <section>
    <table class="text-small">
      <tbody>
        <tr>
          <td>
<b>5</b> - N. to University Area TC</td>
          <td>
            <div class="text align-right" style="color : green;">in <b>5</b> min</div>
        </td>
        </tr>
        <tr>
          <td>
            <b>15</b> - S. to Westshore TC</td>
          <td>
            <div class="text align-right muted" style="color : green;">in <b>7</b> min</div>
          </td>
        </tr>
        <tr>
          <td>
            <b>400</b> - S. to Downtown
          </td>
          <td>
            <div class="align-right muted " style="color : red;">in <b>16</b> min</div>
          </td>
        </tr>
        <tr>
          <td>
            <b>24</b> - E. to Netpark TC
          </td>
          <td>
            <div class="align-right muted">in <b>21</b> min</div>
          </td>
        </tr>
      </tbody>
    </table>
  </section>
  <footer>
    <img src="https://mirror-api-playground.appspot.com/links/icons_30_0025_geo_transit.png" class="left">
    <p>Alumni Dr. @ Laurel Dr</p>
  </footer>
</article>

I welcome any comments, suggestions, feedback on the above! Also, if you're a good designer, please improve the above UI design! :)

[barbeau]

A mockup of the above card layered on top of a photo to illustrate the use case of real-time transit info on Glass from a first-person perspective, via the website Mutual Mobile: http://glasssim.com/?5i06lqm318bs

[evansjoseph]

I'm ready to get started and help you get this off the ground, Sean!

[barbeau]

@evansjoseph Sounds great! I'll contact you offline so we can meet up and figure out the best way for us to get dev access to your Glass device.

[barbeau]

Thanks to @evansjoseph, we now have access to test on Glass at USF.

I've created a demo project here, based on the Mirror API quickstart demo, just to give us a testing ground: https://github.com/CUTR-at-USF/onebusaway-glass-demo

Right now, you can setup and run the project (you'll need to provide your own oauth credentials for Mirror API access - see readme), and on the webpage that is shown at localhost:8080 tap on "Insert sample OneBusAway card", and you should get this view in Glass:

The current arrival info is hard-coded and static, but its a start.

I'll keep everyone posted as I get further on this.

[barbeau]

Oh, and a live version of the above project is at http://transittools.forest.usf.edu, if anyone with Glass wants to try it out.

Instructions:

1. Go to http://transittools.forest.usf.edu/ in your desktop browser
2. Log in with your Google Account you use on Glass
3. Press the “Insert a sample OneBusAway card” button. Hopefully after this step, you should see the real-time arrival info on Glass.

Current limitations:

• Information is static, not real-time, and hard-coded to a single stop. This is meant to be a simple illustration of how the information might look to get feedback.
• Contacts and Subscriptions buttons do not work.

[marks]

This is cool. I have a project at http://askopendata.com [demo at http://www.askopendata.com/demo?question=I%27m%20near%20Union%20Station&contact=Ask%20Metro ]* that uses live data and integrates with OneBusAway. What is needed from the folks here to take this from static prototype to actual live glassware?

*-The reason my app does more than just OneBusAway is that my local metro train operator (WMATA.com) does not do GTFS-RT for trains and so I have to use their API and mash it up with other data from the likes of OBA, bike share locations, etc. Is there a way to do this from the OBA backend so I end up just calling the OBA API as opposed to several APIs perhaps?

Mark

[evansjoseph]

Oh awesome ᐧ

On Thu, Feb 27, 2014 at 2:06 PM, Mark Silverberg notifications@github.comwrote:

This is cool. I have a project at http://askopendata.com [demo at http://www.askopendata.com/demo?question=I%27m%20near%20Union%20Station&contact=Ask%20Metro]* that uses live data and integrates with OneBusAway. What is needed from the folks here to take this from static prototype to actual live glassware?

• The reason my app does more than just OneBusAway is that my local metro train operator (WMATA.com) does not do GTFS-RT for trains and so I have to use their API and mash it up with other data from the likes of OBA, bike share locations, etc. Is there a way to do this from the OBA backend so I end up just calling the OBA API as opposed to several APIs perhaps?

Mark

— Reply to this email directly or view it on GitHubhttps://github.com/OneBusAway/onebusaway-application-modules/issues/60#issuecomment-36278337 .

Joseph A. Evans

PhD Graduate Research Associate| AIST - Alliance for Integrated Spatial Technologies| School of Geosciences | University of South Florida |4202 E. Fowler Ave., CMC102| Tampa, Florida 33620| 813.974.0613 |318.228.5079 | josephevans1@mail.usf.edu josephevans1@mail.usf.edu|www.aist.usf.edu http://www.aist.usf.edu/ |

[barbeau]

@marks Time :). This has been a side-project I've been working on, but its been put on the back burner due to some other priorities. I plan to resume work on it soon, but don't have an exact date/time. @evansjoseph has been kind to share his Glass with us, but we're currently looking to get our own Glass to speed up the development process.

Mobility Lab has an OBA instance for D.C., but I don't believe WMATA is included yet. I know @kurtraschke has been working on expanding OBA D.C. coverage, but don't know where he's at as far as WMATA.

[marks]

@barbeau thanks for the update. I'd be happy to share my experience working on the Metro component of http://askopendata.com for anyone who has questions. You can reach me off of github at mark@socialhealthinsights.com

[barbeau]

@marks Thanks, will do, that would be helpful. Is askopendata.com open-source?

[marks]

@barbeau - NP. It isn't at this point but open sourcing it is not out of the question. A potential hybrid model is to make the data modules (contacts and their business logic in terms of the Mirror API nomenclature) open source and allow people to submit new modules for inclusion.

[barbeau]

Quick update - We got our hands on our own Glass, and I've been working on a GDK (i.e., native Glassware) version of OBA. This allows a more immersive experience than the Cards via the Mirror API, and is more responsive for real-time interaction (including user-triggered voice commands from within Glass). I see the full OBA Glass solution being powered by both Mirror API and GDK, where Mirror API would deliver timely Cards to your timeline, and you could tap on these cards to jump into the native app. And, since I'm primarily an Android developer, I can advance the GDK implementation far faster than the Mirror API implementation :).

Current work on OBA with GDK is taking place in this branch: https://github.com/CUTR-at-USF/onebusaway-android/tree/glass

If you install it (run gradlew installDebug from root of project using command-line, assuming you have Android dev environment setup - or, download the current APK here, and follow these directions to install it on Glass), its activated by using voice commands "ok glass", then "find my bus". Current code will figure out your region (Puget Sound, Tampa, or Atlanta), find the closest bus stop to your real-time location, and then show you the arrival times for that stop (and refresh the times every minute). Navigation in/out of the app after its launched is still a little clunky (i.e., broken) - I'm going to fix this soon. But, I wanted to give early adopters a chance to take a look :).

[marks]

@barbeau - awesome! Agree that a GDK app will have the best user experience. Just FYI, though, Mirror API apps can be triggered by the user using voice commands using contacts and subscriptions.

One request is to make sure that it is easy for people to switch the OBA instance (perhaps use a QR code input via camera or at least make it configurable in the code). I say this because here in the DC area we have an OBA instance that is not connected to the onebusaway.org main site.

Thanks!

[barbeau]

@marks Thanks for the clarification - as you can tell, I'm still a fairly new Glass user :). I did realize as I was typing the above that I forgot to enable "experimental" regions - I'll change this so all regions are included in the next build (including D.C.). I'll look into the Glass QR Code input (same as what's used for WiFi setup) too, since I agree it would be nice to support custom server instances as well (that aren't in the OBA server directory used by the OBA apps).

[barbeau]

@marks Ok, above link to current APK should now support D.C.

[barbeau]

I've fixed the navigation issue in and out of the app, so the current GDK version should be usable (to the extent that the closest stop is the one you're interested in). APK is available at the same link.

Also, I've opened an issue on the onebusaway-android project to track progress of the GDK Glassware (to keep this thread Mirror API-specific in the future).

So, please follow https://github.com/OneBusAway/onebusaway-android/issues/116 for future GDK version updates, and leave feature requests/feedback there.

[aaronbrethorst]

mooted.