Uproot: Reverse engineer phone trees into media-rich flow charts

[lippytak]

BLUF: Uproot turns phone trees into media-rich flow charts Link: Brain dump below + motivating example at http://bit.ly/calfresh-uproot Project Needs: Everything (dev, design, research/user to seed good examples) Status: Pie

Why? Phone trees are one of the primary interfaces for gov. Especially for social services, where many clients don't have routine access to the desktop web. And especially especially for seniors, who prefer audio calls to anything else (source: Jake's stereotypes). The problem with most phone trees is that they suck. Here's what happens when you call the CalFresh service center:

Cleaned up a bit: (click img for interactive version with audio)

Large, complex, confusing phone trees are a horrible and disempowering user experience. The first step to fixing them is to uproot them and give them some fresh air #CitizenOnboardStyle.

PLUS, Tim wants it!!

How? I think Uproot could take one of a few paths:

1. Fully automated phone tree >> flow chart generator This would use Twilio + transcription service (MTurk would do) to automatically crawl through every path of a given phone tree and create a flow chart with node structure + audio segments + text. Input = phone number. Output = flow chart. Not sure if this is feasible. Definitely not feasible for trees that require custom user input ("please enter your case number and press #.")
2. Interactive web+Twilio UI to support building flow chart from phone tree Imagine typing a phone number into the web UI and clicking dial. Then a new node pops up as you hear the audio. You can type the transcription as you listen to the segment on repeat. Then you press a number to keep moving along and creating new nodes as Twilio saves the audio for you.
3. Generic media-rich flow chart Think Timeline.JS for flow charts. Interactive, embeddable flow charts where every node is assumed to have media (audio, video, img, docs, etc.). This is almost certainly a separate project but it should exist anyway...

[fureigh]

Very cool idea!

There are possible huge accessibility benefits — if you have a hearing impairment, phone trees are more intelligible but not much more usable over TDD — but the web access questions hold. Who might the target user(s) be?

[lippytak]

That's a really good question and I don't have a good answer.

To me, this falls into the 'build empathy through experience' bucket. I'll bet most social services have user-hostile phone trees and I'll bet the decision makers don't know. Why would they? What if we did this for every CalFresh program in CA and emailed all the directors a link to their own program's chart?

More generally the audience (the target viewer) would be someone in charge who cares about the user experience. The user (the creator) would be anyone trying to improve the user experience. A client? An advocate? A staffer? Don't know...

[FamiComplex]

A colleague of mine implemented the reverse of this for a web-based call center service. The idea was fairly straightforward - drag the branches around on a flowchart, save the layout and details in JSON for quick retrieval and follow the tree with either recorded messages or TTS.

It sounds feasible, either way.

[migurski]

See also http://gethuman.com

[daguar]

We have a huge need for this right now for understanding the phone trees of California county social service phone lines. Help (and clever hacks) wanted!

[prashtx]

@daguar: have you made progress, or are you still thinking of different approaches that might work? If the latter: I imagine having a person do this realtime would be really taxing. But you could separate the "explore all the branches" phase from the transcription/modeling phase. A (probably) somewhat simple Twilio app could record the call and also record numbers entered along with their timestamps. Then you could separately farm out those annotated recordings to workers or volunteers, who would describe the tree, maybe using YAML.

If you convert the audio file to a video format, you could also use the number-pressed annotations as a subtitles file, and then you get something distributable that helps the transcription process.

If the trees are not gigantic, you could even have the Button Pressing Corps start from the main menu more often than strictly needed, simplifying the job for the Tree Transcription Corps.

You might get some cool metrics out of this, too, if you have timestamps. Average time (per agency) to hear a full list of branches, average cost of punching the wrong number, etc.

[daguar]

@prashtx Still thinking through approaches but we definitely don't want it to be fully manual. The most likely course does seem like an automated-manual hybrid.

We're really taxing our resources on the enrollment side of the problem right now, so we're hoping this sounds interesting enough for someone to take a first stab at an implementation.

[cathydeng]

lurking on this thread, v cool idea! recently found out about audiogrep, a python library that transcribes audio files. if you can find a way to programmatically traverse a tree & record what you hear, this might be useful.

[daguar]

Neat! I do think decomposing the problem into:

1. Phone tree traversal/mapping
2. Audio processing

is a good first step. I'm still struggling on how to implement (1) comprehensively.

[cathydeng]

assuming that automatic transcriptions would be pretty accurate b/c phone recordings are clear, it seems that the challenge in programmatically tackling (1) comes down to parsing the processed audio - at each node, figuring out what the possible next options are, or if it's a dead end.

this seems like something that could be hacked together (e.g. a naive search for the phrases 'press one', 'press two', etc) since phone recordings have fairly standard language.

[daguar]

I've started a little experimentation / feasibility spike over yonder https://github.com/codeforamerica/uproot

[evanwolf]

There was a startup about ten years ago that made a business out of mapping customer service phone trees and helping customers use the web to navigate straight to the point in the tree where they wanted to be. $ was from selling UX stats to IVR users to improve their phone trees. They built innovative tech but those initial business models never took serious hold.

Their tree mapping model blended automation with crowdsourcing.

[daguar]

I've got some decent progress started in the repo. Here's a screenshot:

[daguar]

Try out the silliness and watch it almost certainly break: https://uproot-dev.herokuapp.com/

[RommelTJ]

If the purpose is to provide a better UX, I'm not sure providing a graphical interface for a deep hierarchical menu of options is the best way to move forward. I guess I'm still struggling to understand who is this for? Senior citizens? What they want is to talk to a human so I'm just not following.

You say, "the first step to fixing them is to uproot them and give them some fresh air", but why? Why is that the first step is what I don't understand. Telephone Systems can be a UX nightmare because the user frequently has little domain knowledge and the service, especially for government programs, is simply too broad. There are simply too many services hidden behind one telephone number and so the trees become too broad and the choices get complicated. It seems to me that what you want is to narrow the choices, not uproot them... I'm probably missing something here, so I apologize.

Without understanding the problem correctly, I would suggest the Sponsor-Selector pattern.

The Steps would be: 1) Determine all Resources available in a telephone tree. 2) Define a Sponsor for each Resource that contains knowledge of when this resource is appropriate. 3) Define a Selector that contains knowledge of all Sponsors and which one is appropriate for a given situation. 4) The Client requests to the Selector when it needs a Resource and invokes the Resource that is returned by the Selector.

Here is an example for a Medical Diagnostic System:

[civicissuebot]

Hello! This issue looks like it still needs help! It's been clicked on 27 times through the Civic Issue Finder on http://www.codeforamerica.org/. Can this issue be closed or does it still need some assistance?

If you wrote this issue, you can always update the labels for specifying tasks, add more info in the description to make it easier to contribute, or re-write the title to make more contributors interested in helping out. If you are an open source contributor, ask and see how you can help by commenting or check out more open issues in this repo at https://github.com/codeforamerica/project-ideas/issues.

Just doing a little :seedling: open source gardening :seedling: of Brigade projects! For more info/tools for creating civic issues, check out Got Issues Thank you!