search

OneBusAway / onebusaway-alexa

An Java-based app to communicate with Amazon Alexa for devices such as the Amazon Echo
Other
52 stars 18 forks source link

Bootstrap UX - Initial selection of a stop #1

Closed barbeau closed 8 years ago

barbeau commented 9 years ago

When the user first launches the skill, we need to determine, at a minimum:

  1. What region they are in (list of regions is available from the OBA Regions API) - If we know which region they are in, then we know the URL for the local OBA API server to request information for - for example, api.tampa.onebusaway.org for Tampa, and api.pugetsound.onebusaway.org for Puget Sound.
  2. What stop they want information about - After we know the region and the local OBA API URL, we then need to know the agency_id and stop_id to get the arrival info for the right stop from arrivals-and-departures-for-stop API (format is arrivals-and-departures-for-stop/<agency_id>_<stop_id>.json). I believe we'd need to use the OBA agencies-with-coverage API to go from a human readable agency name to the agency_id. The user facing stop numbers are stop_code in OBA, so we'd need to go from a human readable stop_code to stop_id using stops-for-location API, with the query parameter set to the stop_code.

Limitations:

Questions I have:

  1. How much of the dynamic info from the APIs needs to be hardcoded for speech recognition? - The Alexa examples all suggest including a lot of utterences so it can understand the user. If we ask the user what city they are in, will Alexa understand city names without having to hard-code them in utterences? Same goes for stop names - I'd be surprised if we could match stop names, given the large number of stops. Numeric IDs do seem to be understandable without hard coding utterences.
  2. What does the intial UX look like? - To find what region the user is in, we need them to tell us something about their location. Possibilities include city name (pending above question) or zip code. Then, they could speak the stop code they are interested in.
  3. Could we link OBA app cloud saves to Alexa? - A very simple way to bootstrap stops for a user that wouldn't require location info would be to simply have Alexa download starred stops from the OBA Android / iOS apps which have previously been saved to the cloud (see https://github.com/OneBusAway/onebusaway-android/issues/32 for Android - cloud saves still to be implemented). However, it's not clear to me how the account linking would happen. All we have is an anonymous ID from Alexa, and the onebusaway-alexa webapp would need a way to authenticate with cloud storage provider (e.g., Google) and tie that info to the ID.
barbeau commented 9 years ago

Re: cloud saves - you can now link a Google calendar account within Alexa app so Alexa can have access, but to my knowledge this isn't available to third party skills. Looks like this has changed - see comment below, and this link.

JamesCMcMillan commented 9 years ago

What region they are in

Making this a decent UX might require:

What stop they want information about How much of the dynamic info from the APIs needs to be hardcoded for speech recognition?

These may be made easier by handling the issue prior to runtime. Things like agency lists, the areas they cover, the stops they have, and the routes they have are slow-changing and could be cached in a database. Potential gains:

Another application for pre-generation (database is nice here but isn't necessary): since we can get a complete list of stops for every agency, we may be able to generate a complete list of sample utterances for stops. This might be much friendlier than requiring a stop code. I'm not sure if there's a limit to the number of sample utterances you can register. Conceivably even for non-hits on the sample utterances we might be able to fuzzy-match the text with the list of stop names.

Could we link OBA app cloud saves to Alexa

Cool potential feature, but also a potential rabbit hole. I'd probably worry about this later.

barbeau commented 9 years ago

According to this SO post, the Google Geocoding API will accept just a zip code and return a lat/long, so we could use this to resolve the closest region based on zip code.

barbeau commented 9 years ago

For v1 of this application, here's the bootstrap UX I'd like to implement:

  1. Ask the user their zip code city (see comment below) - we then use the Google Geocoding API (and OBA Regions API response) to determine their closest region.
  2. Ask the user their stop number (i.e., stop_code in GTFS) they want to set as their "home stop" - we then use the region stops-for-location API to get the stop_id for this stop.

After the above interactions, we'd persist the following information for this user:

Then, the next time Alexa triggers a request, we look up their info using the userId, and we can use the stop_id and region ID to query arrival times for their home stop, and read off arrival info. We should cache the Region base URL so we don't need to hit the Regions API again when the user requests arrival info - however, we should refresh the base URL info from the Regions API occasionally to reflect any changes in region setup (we refresh this weekly on OBA Android).

The main advantage of the above is simplicity of the implementation - we stick to numbers as input and the built-in city slot, which Alexa should be able to recognize without needing sample utterances. It also doesn't require caching any OBA data, and only requires a single REST API request to fetch arrival info. We can build on this and add more complexity after its up and running. We might need to do some disambiguation in case the stop_code matches more than one stop - this is possible when multiple agencies are included under the same OBA region (currently the case in Puget Sound, and soon in Tampa).

Apparently Amazon DynamoDB supports a DynamoDB Java Object Persistence Model, which looks like a good candidate for a persistence store.

On a related note, I found this Alexa page today that discusses linking accounts via access tokens and OAuth, which can be used to authenticate with other systems. I believe this is new, and would allow the case of pulling cloud saves, for example authenticating against Google cloud storage. But, let's save this for later.

barbeau commented 8 years ago

Here's what the dialog would look like on first startup:

User: Alexa, start OneBusAway

OneBusAway: OneBusAway. What is a zip code near you? OneBusAway: OneBusAway. What city are you located in?

User: Tampa

OneBusAway: You can find your bus stop number on the sign near your stop. What is your stop number?

User: 3105

barbeau commented 8 years ago

There is now a built-in city slot that we could use instead of zip code: https://developer.amazon.com/public/community/post/Tx4DUAGXZXZ5A6/Announcing-New-Alexa-Skills-Kit-%28ASK%29-Features:-Built-In-Intents-and-Slot-Type-f

We would use this as input to the geocoder, to get a lat/long relatively near the user. I have a feeling city name may be more natural to users that zip codes, so let's try city name.

One caveat with city that is that we can't use it to match against OBA region names. For example, the OBA region covering Seattle is "Puget Sound" - Seattle isn't actually in the name.

Another caveat is that Amazon only recognizes city names with a population over 100,000 - we may need some error handling for unrecognized city names that asks again for the "largest city near you".

barbeau commented 8 years ago

I've written the code necessary to go from an initial "city" input down to getting a bus stop by ID in this Lambda function: https://github.com/OneBusAway/onebusaway-alexa/blob/master/src/org/onebusaway/alexa/LambdaFunctionHandler.java

This uses the onebusaway-client-library to access the OBA REST APIs, which I created for this project (but could be used in any Java-based project).

This code now just needs to be organized into the Alexa lifecycle, and the data from the user needs to be persisted on initial setup.

I've started work on the persistence code using Dynamo Db in this package: https://github.com/OneBusAway/onebusaway-alexa/tree/master/src/org/onebusaway/alexa/storage

This is patterned after the ScoreKeeper skill example. ObaDao is the main class used by other classes to read and write data to Dynamo Db.

greenwoodcm commented 8 years ago

With the OAuth login you could potentially provide a web UI to bootstrap a user's account, that might be overall easier for the user than providing all the configuration via voice. So the flow I would envision:

1) a user goes to alexa-oba.com or whatever 2) creates an account 3) on the web, sets their home stop, preferred routes, etc. 4) logs into the alexa app with their same credentials 5) each invocation of the lambda function provides the authenticated user's ID, allowing you to query their settings on the backend

On a related note, I'd be interested in contributing here, I've written a few alexa apps just to try it out, and also have a lot of experience with AWS, including DynamoDB. Let me know where you could use contribution.

barbeau commented 8 years ago

@greenwoodcm Agreed, the OAuth piece could definitely simplify initial setup of user stops. Ideally I'd like to be able to link to OBA Android and iOS apps so users could make their starred favorite stops there automatically available for Alexa. Feel free to open a new issue for OAuth specifically - this wouldn't be v1, but definitely could be in a future release.

And we'd definitely welcome contributions! There should be a large chunk of contributions coming shortly that moves the app to a fully functioning Alexa skill, which should fulfill most of the v1.0 features (I believe including writing/reading data to/from DynamoDB). You can either wait for that to land, or start fleshing out some of the > v1.0 release features. As a heads up, you'd need to sign the online OBA ICLA before we can merge contributions.

philipmw commented 8 years ago

Hi, team. I am working on a first draft of this now. I hope to have a PR out in the next day or two.

philipmw commented 8 years ago

Happy to report that I am making good progress. Code is pushed to my repo, city branch. I hope to wrap it up in the next few days.

barbeau commented 8 years ago

Cool @philipmw! I hope to take a look soon.

BTW, I saw today that the new Uber skill allows the user to set their Echo location via the Alexa app, and then use this location in Uber skill. From some quick Googling I didn't see anything in the third party skill docs for this so I'm not sure its exposed to all skills yet.

barbeau commented 8 years ago

I'm closing this issue, as the core functionality is now mostly implemented following this design (via https://github.com/OneBusAway/onebusaway-alexa/pull/8 and https://github.com/OneBusAway/onebusaway-alexa/pull/21). I've split out the OAuth/cloud saves into a separate issue in https://github.com/OneBusAway/onebusaway-alexa/issues/16, currently targeted for v1.1.