Promptly + Connect = POWER UP ACTIVATED

[lippytak]

Promptly texts a few hundred (right?) SF CalFresh clients each month who are at risk of getting discontinued. Lots of these clients immediately call the HSA service center to figure out watsup and presumably fix something with their case. BUT it's looking like lots of these calls don't make it very far. Here's the distribution of call duration:

This may partially explain why Promptly doesn't seem to be reducing churn much if at all. One possible reason the phone calls are so short (and presumably so useless) is that people either don't want to go through the phone tree, hear the hold time and hangup, or start waiting on hold but don't make it to the end. We can solve ALL of these problems with DRUM ROLL Connect (per @ttmp's idea)!

I think all this would involve is changing the voice_url for the Promptly production number to point to the Connect app. @daguar is that right? Then any incoming calls would hear Dave's pretty voice saying they can hang up and get on with their day.

Issues:

• Testing: We have 0 active users for Connect so we would need to test it (preferably) with real users first. We'd like to start with a few CBO/advocates before giving it to clients.
• Language: Connect only goes to the English line right now. I don't remember if Promptly prod has one phone number or unique numbers by language? If it's one phone # across languages this might be an issue. @andyhull can you confirm?
• What am I missing?

cc @rduecyg and @RebeccaCoelius

[daguar]

I don't know how Promptly works (ie, if the voice_url config is how we do it) but, yeah! We could certainly try Connect.

Big :+1: to doing more testing with Connect in the wild. Specifically:

1. We should confirm that the hangup rate is fairly low — even by testing this a lot we could potentially acculturate the agency staff to using it, and reduce the hangup rate. But we have very little data just yet.
2. We should do real significant user testing with clients to make sure this makes sense and is convenient for them. We may potentially want to give them the option to just call the normal line.
3. Ideally, we should randomly assign the first group of users to see if there's any effect from the use of Connect.

On Thu, Nov 6, 2014 at 4:13 PM, Jake Solomon notifications@github.com wrote:

Promptly https://github.com/codeforamerica/promptly texts a few hundred (right?) SF CalFresh clients each month who are at risk of getting discontinued. Lots of these clients immediately call the HSA service center to figure out watsup and presumably fix something with their case. BUT it's looking like lots of these calls don't make it very far. Here's the distribution of call duration: [image: eep] https://cloud.githubusercontent.com/assets/2533112/4945904/2603d46c-660d-11e4-86f3-8d62270ca91d.png

This may partially explain why Promptly doesn't seem to be reducing churn much if at all. One possible reason the phone calls are so short (and presumably so useless) is that people either don't want to go through the phone tree, hear the hold time and hangup, or start waiting on hold but don't make it to the end. We can solve ALL of these problems with DRUM ROLL Connect (per @ttmp https://github.com/ttmp's idea)!

I think all this would involve is changing the voice_url for the Promptly production number to point to the Connect app. @daguar https://github.com/daguar is that right? Then any incoming calls would hear Dave's pretty voice saying they can hang up and get on with their day.

Issues:

• Testing: We have 0 active users for Connect so we would need to test it (preferably) with real users first. We'd like to start with a few CBO/advocates before giving it to clients.
• Language: Connect only goes to the English line right now. I don't remember if Promptly prod has one phone number or unique numbers by language? If it's one phone # across languages this might be an issue. @andyhull https://github.com/andyhull can you confirm?
• What am I missing?

cc @rduecyg https://github.com/rduecyg and @RebeccaCoelius https://github.com/RebeccaCoelius

— Reply to this email directly or view it on GitHub https://github.com/codeforamerica/connect/issues/29.

Dave Guarino Engineer, Health (2013 Fellowship alumnus) Code for America http://www.codeforamerica.org/ dave@codeforamerica.org LinkedIn http://www.linkedin.com/in/daveguarino/ | GitHub https://github.com/daguar/

[andyhull]

I was just thinking about doing this a couple of weeks ago. I even looked at the code for a minute. I think some user testing would be great. Also we need to get to the bottom of the calls. Are people really not getting through? Do they hear the wait time then hang up? How many of those people actually call back? We don't have any reliable data on this.

I think integrating connect could be really awesome. If a user calls the voice_url will they automatically be in line to get a call back from the call center/wherever? How does this actually work?

@lippytak you are correct that there is only one number for each program. We don't break this down by language, but you potentially could by creating multiple organizations. Could get a little difficult to manage though.

[alanjosephwilliams]

We are going to move forward with a user testing this Saturday (a week from tomorrow) at the SF Central Library. If we have a working prototype, we could conceivably include it in the slate of testing. Otherwise we can discuss how to collect some actionable data.

[rduecyg]

Back of napkin calculation: Number of clients that likely talk to someone equals the number of people using Promptly who renew their benefits. This could be a coincidence, but regardless it highlighted to me the need to use a funnel as a mental model.

1(Call back rate 35%)(Talk to a Person Rate 42%) = 14.7% Estimated renewal rate using Promptly = 15% Estimated renewal rate for all discontinuances = 13%

If we can bump up the "call back rate" and the "talk to a person rate" it could make a big difference in the funnel. 1(Call back rate 50%)(Talk to a Person Rate 75%) = 37.5%

Right now that difference means hundreds of people, but as more people sign up for Promptly that could be thousands.

On Fri, Nov 7, 2014 at 8:20 AM, Alan Joseph Williams < notifications@github.com> wrote:

We are going to move forward with a user testing this Saturday (a week from tomorrow) at the SF Central Library. If we have a working prototype, we could conceivably include it in the slate of testing. Otherwise we can discuss how to collect some actionable data.

— Reply to this email directly or view it on GitHub https://github.com/codeforamerica/connect/issues/29#issuecomment-62169167 .

[lippytak]

Ahhh...that napkin looks promising.

@andyhull here's the UX of Connect right now:

1. User calls the Connect phone #
2. Recording of Dave tells them to hang up and we'll call them back
3. When EW answers the phone, they hear Leo O's voice giving a bit of background and prompting them to Press 1 to talk to the client. Hopefully they press 1 and this initiates the call. Sometimes they don't and this hangs up, which skips us to step 5 below.
4. Client's phone rings with same number they initially called and the two chat.
5. They hang up. A few seconds after hangup, the client gets another phone call from the same # that asks them to answer one feedback question.

Everything is in English on both the client and EW side. @andyhull I encourage you to try this out using our dev number: 415-969-2300. Note you'll actually get a call back from a CalFresh EW...(hopefully)

Here's what I propose as next steps (very open to feedback!):

• @daguar @alanjosephwilliams & myself will take on testing + user testing to confirm:
  • Reasonable hang up rate (aka when EW hangs up before they talk to the client)
  • Users understand and like what is happening
• Postcode team does basic research on the current incoming calls to make sure this is worth doing (see @andyhull's thoughts above)
• @andyhull figures out the best way to expose a small subset of English speaking Promptly users to Connect. Connect could be on a separate phone number or we could point the Promptly voice_url to Connect, though neither approach is perfect. I wouldn't want to do much more than ~20 Connect calls the first time, which suggests we should only expose it to ~50 clients or less assuming 40% call back rate.

One last though: given that Promptly initiates contact via text and lots of people reply unprompted, it might work really well let users initiate the HSA call via a text, too. Something like: "We can't reply back via text right now, but reply 'CALL' and we will call you when a representative is free."

[rduecyg]

Just FYI, but PostCode is pretty slammed with work until the mid-December.

[daguar]

Thanks Reed. We're going to be testing Connect further anyway to get real connection rates and CBO user feedback. I don't expect we'll be ready to try a Promptly integration until at least beginning of January.

On Nov 10, 2014, at 2:28 PM, Reed notifications@github.com wrote:

Just FYI, but PostCode is pretty slammed with work until the mid-December.

— Reply to this email directly or view it on GitHub.