Rewrite of the in-app onboarding flow

[JGreenlee]

As we begin to rewrite the onboarding flow, we should first re-evaluate the design and layout to make it more accessible and user-friendly.

Yesterday, we were walking a user through the onboarding process and observed a few issues. They were using a screen reader (VoiceOver on iOS) to navigate through the app and could not enable the necessary permissions from the app. The current release of the app has permissions organized into expandable sections, none of which the user could open using a screenreader. We are already reconfiguring the permissions screen (https://github.com/e-mission/e-mission-docs/issues/919), and the new version will be a single list without expandable sections. We may want to prioritize implementing this on the onboarding flow sooner than previously anticipated.

Although the user was not able to enable permissions, they were able to navigate past the permissions page and ended up on a screen that was not yet enabled. After this, there was no way for the foreground to go back to the permissions screen - the user was only selecting elements in the background.

I think we should generally reconsider how the navigation between screens works in the entire onboarding flow. The current implementation uses an Ionic mechanism called ion-slides. All of the slides are constructed upfront so they already exist in the DOM and just aren't visible. This is problematic because we don't want people to be able to progress past a screen until they have completed the required task for the current screen.

It is clear that we should move away from ion-slides ASAP because it is not accessible – but I think we should also take some time to think about what the ideal design will look like, instead of just reaching for the nearest accessible replacement.

[JGreenlee]

The goals of each screen in the current onboarding flow:

1. Welcome the user to the app and allow them to either 1) scan a QR code or 2) paste a code as text
2. Display a summary of the program/study
3. Display information about what data is collected and how it is used, and ask for consent from the user
4. Guide the user through enabling the necessary permissions
5. Give the user an opportunity to save their OPcode -- For programs with assigned tokens, this step does not seem necessary. Users should have already been sent their code (presumably by email?) and they can go back and reference it at the source if they need to -- For open studies / not assigned tokens, we could instead ask people to save the OPcode from the website join page and provide a download button to save as PNG or PDF. This way they have their OPcode safely saved somewhere before they even need it in the app. To me, this seems safer and more intuitive, maybe resulting in fewer 'ghost' users
6. (if a previous response exists) Ask users if they want to edit their demographics responses -- Is this even needed? Demographics responses can already be edited from the Profile, so why not just always use the previous response if it exists?
7. If necessary, show the demographics survey to be completed or edited

---

Let's think about how we can combine some of these pages or incorporate them in a different way

(1) has to be its own page, because everything from that point on is dependent on what OPcode was used. (2) and (3) can be shown on the same page. It might be kind of long to scroll through, but it's already long anyway. People are used to long 'terms of use' statements - this is kind of like that. (4) should just be a series of popups and not need its own screen. We are working on that in https://github.com/e-mission/e-mission-docs/issues/919. (5) is not necessary for many programs, and potentially could be eliminated for open studies as well if we rework join page (6) may not be necessary (7) would be shown if it's the first time through the onboarding flow

[JGreenlee]

If we can apply the above suggestions, I think we can do this in 4 steps:

1) Welcome and ask for scan/paste OPcode 2) Summary of the program/study at the top, scroll down to see data collection info, then "Accept"/"Refuse" at the bottom 3) Popup new permissions UI 4) Show demographics survey (if it's the first time through onboarding)

[Abby-Wheelis]

For open studies / not assigned tokens, we could instead ask people to save the OPcode from the website join page and provide a download button to save as PNG or PDF. This way they have their OPcode safely saved somewhere before they even need it in the app. To me, this seems safer and more intuitive, maybe resulting in fewer 'ghost' users

Are some of those random opcodes for the individuals once they scan? So would that download be for the study's opcode and not their individual one? If it's a single opcode for the study and then individuals are identified by random codes then I feel like we should absolutely encourage saving them, since there isn't a way to recover that random code. I do, however, keep getting myself confused over how the opcodes work, so I could be wrong here.

[JGreenlee]

Are some of those random opcodes for the individuals once they scan? So would that download be for the study's opcode and not their individual one?

It used to be like that. They would scan the study's QR code and then their user-specific token was generated on the phone. But now we have it all in one QR code, and the generation actually happens on the website join page.

[JGreenlee]

If you refresh https://open-access-openpath.nrel.gov/join a few times, you can see that the randomization is actually happening there.

So I would suggest we urge people to save it from there, where it is generated, not during onboarding. (But if for some reason they do need to save it from in the app, they can also do that through Profile > Share QR)

[Abby-Wheelis]

Ah, ok, I think I get it now. I definitely saw that code when I was dynamically configuring the languages but didn't quite put two and two together. Thank you for explaining. It definitely makes sense to save from the join page then, and hopefully we could download the qrcode and the text together into one pdf (which currently the app does not do - if you "save to files" it will save them as two documents though, so the function is there, just not as elegant)

[JGreenlee]

@rahulkulhalli Including you in this discussion as well -- this is another area of the UI we are overhauling and aiming to improve UX in the process.

[shankari]

I upvoted this because I think it is important to allow users with screenreaders to be able to at least install the app.

@JGreenlee any high level visualizations/wireframes of what the proposed flow might look like?

[JGreenlee]

@shankari Which part of it do you upvote?

Specifically, I'm looking for confirmation on:

1) removing the "Save OPCode" screen and instead urging people to save from the join page

2) removing the screen that prompts whether to edit the demographics survey response – if a response already exists, simply skip

If these are fine, I can draft up some wireframes for the 4 screens that we will need.

[shankari]

(5) is not necessary for many programs, and potentially could be eliminated for open studies as well if we rework join page

If you see the blame, I added this primarily as a way to cover up the slow load time of the demographic survey. Because the demographic survey is fairly long, and we store both XML and JSON, it can be noticeably slow to load. I wanted to give people something to do while it loaded in the background, so I made the previous popup into a little interstitial.

If you weren't planning to combine consent and permissions, we could try loading after the consent page as well. But we cannot try to load before the consent page, since we cannot register the user before they have consented, and we cannot try to load an existing survey for an unconsented user.

Right now, we actually only register the user after they have consented to the terms and turned on all permissions. This is to avoid having "ghost" users from people who started the install process but balked at the permissions. So I think that we should continue to register only after permissions, which means that it would still be nice to have something for people to do while waiting for the demographic survey to load.

Open to other suggestions on what this might look like...

For the record, we haven't actually received any pushback on this from any partner. The one piece of partner-specific feedback that we have received is that instead of "share", we could just tell people to take a screenshot because the menu opened up by "share" is confusing. I personally think that "share" is fine, because I like to send email; would appreciate thoughts from others.

(6) may not be necessary (7) would be shown if it's the first time through the onboarding flow

Yes, people can always go and edit the demographics from the profile. But from a behavior perspective, the chances of people, even well-intentioned people, doing that are approximately zero. This is an option to at least remind people when they change phones that their demographics may have changed and maybe they can tell us what they are. So that when we generalize from this sample to a broader population, we are less off. This is particularly true for data collection efforts that are longer than phone replacement cycles (> 1 year ).

[shankari]

@JGreenlee

Which part of it do you upvote?

Just the general concept of rethinking the onboarding flow and making sure that it is accessible as well https://github.com/e-mission/e-mission-docs/issues/956#issuecomment-1694181268

I was added my comments on everything before (5), (6), (7) earlier, and on (5), (6), (7) today.

[JGreenlee]

If you see the blame, I added this primarily as a way to cover up the slow load time of the demographic survey. Because the demographic survey is fairly long, and we store both XML and JSON, it can be noticeably slow to load. I wanted to give people something to do while it loaded in the background, so I made the previous popup into a little interstitial.

If you weren't planning to combine consent and permissions, we could try loading after the consent page as well. But we cannot try to load before the consent page, since we cannot register the user before they have consented, and we cannot try to load an existing survey for an unconsented user.

I am not suggesting to combine consent and permissions. I suggested to combine the program/study summary text onto the same page as the consent text. But the permissions popup won't launch until after the user has clicked "I agree". So can't we load the survey while the users are interfacing with the permission popup?

Because the demographic survey is fairly long, and we store both XML and JSON, it can be noticeably slow to load.

I just checked the XML and it's 110 kb - the JSON is 122 kb. It really shouldn't be that bad from a networking standpoint. Is there something causing the retrieval to be slow on the server? Also, I just checked what would happen if we gzipped the files. That brought them down to 11KB and 14KB respectively. So if the filesize is the issue, there are ways we can deal with that too.

Also, we hopefully we won't even need the JSON once we get enketo-transformer implemented.

[JGreenlee]

I am trying to have onboarding be as minimal as possible because I think the long onboarding process is the worst part of the current UX. It is also a vitally important part because people will give up if it takes too long.

So I am trying to be fairly thorough in considering all the possible optimizations we could make.

[JGreenlee]

Yes, people can always go and edit the demographics from the profile. But from a behavior perspective, the chances of people, even well-intentioned people, doing that are approximately zero. This is an option to at least remind people when they change phones that their demographics may have changed and maybe they can tell us what they are. So that when we generalize from this sample to a broader population, we are less off. This is particularly true for data collection efforts that are longer than phone replacement cycles (> 1 year ).

This seems strange to me, though, because then we have updated demographics for people who got a new phone, but we have outdated demographics for people who are still using the same phone. Doesn't that introduce a different kind of bias?

If collection lasts for over a year, it would be better to prompt everyone at the same time.

[shankari]

I am not suggesting to combine consent and permissions. I suggested to combine the program/study summary text onto the same page as the consent text. But the permissions popup won't launch until after the user has clicked "I agree". So can't we load the survey while the users are interfacing with the permission popup?

The load here is not of the survey template - for the vast majority of partners, the survey is loaded locally anyway since it is not overridden. The load here is of any existing survey results that the user has. To load the existing survey results, we need to have the user to have signed in to the server and created an account if necessary. Right now, we create the account after the permission screen because we don't want "ghost users" who are intimidated by the permissions.

So if we want to load the existing survey, we must do so after the permissions.

If we don't load the existing survey, we don't know whether we need to ask users to fill in a survey or give them the option to skip. Even if we don't care about updated demographics per question below (for which I don't know the answer):

This seems strange to me, though, because then we have updated demographics for people who got a new phone, but we have outdated demographics for people who are still using the same phone. Doesn't that introduce a different kind of bias?

we will still need to know whether to skip the demographic survey or not depending on whether the user has already filled it out before or not

This is one of the feedbacks that we did get from the field - back when we used Google Forms for the survey, people complained that they were forced to fill in the survey every time they switched phones. So it is a requirement that if the user filled out a survey before (aka has an existing account), then we must at least give them the option to use it.

[shankari]

Doesn't that introduce a different kind of bias?

Check with Andy or somebody else with a social science background or what is better.

[shankari]

One idea that has been discussed before is to basically just drop the forced initial demographic survey completely and to instead prompt people for parts of the demographic survey through the first month or something. I believe there was some other app that partnered with NREL that did this sometime.

I am really concerned that by breaking this up, we will just not get all the demographic information, but I leave it open as a thought

[shankari]

I think that one of the challnges here is that the onboarding flow is so important that changing it feels really risky. We know that the current onboarding flow works reasonably, as evidenced by multiple ongoing programs, including one that is primarily composed of 55+ adults.

I would like to proceed with caution, in stages, here:

• make the changes that we all agree on (e.g. try to "don't optimize" permissions)
• make changes for accessibility so we can unblock UNSW

Plan changes based on:

• behavior theory: back up the suggestions on changes with theory that shows why it should work, and/or
• partners who believe the theory and are willing to try out the changes for their study/program and/or
• user interface testing with a sample population and a clear idea of how we would recruit testers

We should at least have the list of the more extensive changes identified and backed with theory so that we can pitch them to partners. Then we can enable them through the config, and then finally potentially make them the default.

[JGreenlee]

I would like to proceed with caution, in stages, here:

• make the changes that we all agree on (e.g. try to "don't optimize" permissions)
• make changes for accessibility so we can unblock UNSW

Sounds like a plan. @Abby-Wheelis and I can probably start on this tomorrow or Thursday, keeping the same layout and structure, but rewriting in React and giving consideration to accessibility. We'll also instrument onboarding time so that we can quantitatively compare this flow with potential future iterations.

[Abby-Wheelis]

I'm working on incorporating the consent page in the re-write, specifically how to combine the summary and privacy/consent pages. From earlier comments, it looks like we want to include the summary, then the privacy policy, then the consent buttons in one long scroll. I have the content in, but it needs formatting.

These are the two sections so far -- I'm looking for input on arrangement/formatting. I'm thinking of making "Development environment (study)" a title as the current "NREL ... OF USE" is, so we'd have two major sections. I could also see a case for only having the study name as a more major title, then the "NREL ... OF USE" as a slightly more minor title.

Since these are being combined, do we want the summary section added to the popup in profile as well?

[JGreenlee]

I agree with giving the 2 major titles equal precedence. You might also try centering them and see how that looks.

I also imagine a divider (ie <Divider /> from React Native Paper), with a bit of vertical margin, could help separate the sections visually.

[Abby-Wheelis]

Here's what it looks like now, in onboarding and in profile:

I think the centering helps set the titles apart from the sub-headers, and the divider is nice also, thanks for the suggestions!

[JGreenlee]

Adding the survey page went more smoothly than expected. All of the Enketo surveys are now displayed using EnketoModal so we can remove a lot of the old Enketo code.

Permissions still need some work - the AppStatusModal should show up after the user agrees on ConsentPage, but it does not currently.

[Abby-Wheelis]

Permissions still need some work - the AppStatusModal should show up after the user agrees on ConsentPage, but it does not currently.

I noticed that, working on it right now. I'm noticing that the setStates for "Platform" and "os version" that I have as a part of the setup are not setting the state (at least not in time) so the app thinks that we're on an unsupported platform, and never sets up any permissions.

[Abby-Wheelis]

I ended up bypassing using state for "platform" and "osver" and just queried them from "window" each time they're needed in setup (we check to make sure window[device] is defined before setting up permissions). We'll also have the chance to move away from "window" and use Platform.OS, it returns "web" now, but could be used once it's fully a Native app. I decided not to use state after reading that set does not change state in the running code After that change, the permissions is popping up for me right after I hit "I accept" as I land on the dashboard screen (survey page now that I pulled that change!). I also just updated the modal so it is inescapable until the overallStatus is resolved.

[JGreenlee]

The new onboarding flow (https://github.com/e-mission/e-mission-phone/pull/1032) is technically functional right now -- but, as I've realized by reading back through this issue, it's not functioning in the desired manner.

Right now, we create the account after the permission screen because we don't want "ghost users" who are intimidated by the permissions.

So if we want to load the existing survey, we must do so after the permissions.

The current version of the new flow does not do this -- it registers the user as soon as they agree to the consent page. Assuming we still anticipate 'ghost users' who consented but quit during permissions, we still need to do some additional work to rearrange this.

However, I would like to note that the current iteration does appear to provide a better UX. The survey loads in the background while the user is working on permissions, and there is no a need for a 'filler' page. So I still think we should revisit this design later.

[JGreenlee]

Demo of our progress on the new onboarding flow:

https://github.com/e-mission/e-mission-docs/assets/15843932/1a11b15b-7adb-4c55-aa01-cd254396cc32

And if a demographics survey response already exists, this shows instead of the survey automatically popping up

---

Still TODO:

• Add back the app icon on the Welcome page
• Implement QR scanning
• Start pre-loading the survey on the "Save QR" page
• Make the survey not dismissable if it's the first time taking it
• Potentially redesign the welcome page layout?
  • The very first choice the user must make is "Scan QR" or "Paste text". I think a side-by-side layout, with large descriptive icons, could be more effective.
  • Is this something we'd be interested in considering? If so, I'd love to wireframe
• strip out ALL the old 'intro' code!

[shankari]

@JGreenlee now that I see it, I am concerned that the combination of the summary and the terms into one page leaves us with a "wall of text". From my recent experiences with setting up phones and apps, pagination during onboarding is fairly common, with the first page being some kind of high-level summary/intro.

Can we get some feedback from the rest of the UI team on pagination of summary versus terms?

potentially redesign the welcome page layout? Is this something we'd be interested in considering? If so, I'd love to wireframe

Totally supportive of this. I wrote the current welcome page (https://github.com/e-mission/e-mission-phone/commits/master/www/templates/join/request_join.html), albeit with some recent feedback from you on colors etc so it would almost certainly benefit from a redesign.

[JGreenlee]

Thoughts on this wireframe?

[Abby-Wheelis]

That looks really great! I like the two main options, and I think it makes the landing page clear and inviting. One clarifying question, are we keeping the little "info" button that had "tips" for correct operation? It looks like the NREL OpenPATH title might be clickable, is the popup accessed that way?

[JGreenlee]

Oh, interesting! I never even knew that was clickable because it didn't look like a button to me.

I think it wouldn't hurt to have a ⓘ button, separate from the "Welcome" text and styled as a button so it's clear that it can be clicked. The upper-right corner seems like good spot for it.

[JGreenlee]

Added the button, tweaked colors, and adjusted the text to be more user-friendly. Unless anyone else has feedback, I think this is ready to be realized as code!

[shankari]

@JGreenlee

Before I go to review this, any updates on the first part of https://github.com/e-mission/e-mission-docs/issues/956#issuecomment-1732335053

now that I see it, I am concerned that the combination of the summary and the terms into one page leaves us with a "wall of text". From my recent experiences with setting up phones and apps, pagination during onboarding is fairly common, with the first page being some kind of high-level summary/intro.

For the record, I took this video of my installing a new location-based app (Gig, which is a floating carshare program in the Bay Area and Seattle). As you can see, they do use pagination during the onboarding process. In fact, they use a page for every single permission that they request.

https://github.com/e-mission/e-mission-docs/assets/2423263/9ba00a9d-494e-46a4-bf32-7cd0a9a05301

Similarly, they use pagination for the steps related to license validation (which I am not going to upload a video of). So I think that a pagination/"wizard" workflow is considered reasonable when you want the user to perform a sequence of steps.

[Abby-Wheelis]

now that I see it, I am concerned that the combination of the summary and the terms into one page leaves us with a "wall of text"

One of the pros of having the two together is that the summary text is now visible in the popup on the profile screen alongside the terms. If we were to split them up, would we want a different way to see the summary from profile, or do we only want it to be viewable once?

I believe the rest of onboarding is accessible from profile: demographics survey, opcode, terms, and appstatus/permissions each have their own setting rows.

[shankari]

One of the pros of having the two together is that the summary text is now visible in the popup on the profile screen alongside the terms. If we were to split them up, would we want a different way to see the summary from profile, or do we only want it to be viewable once?

IMHO, it is fine if the summary is only viewable once. It is really intended to orient people so that the first screen that they see after "joining" is not a wall of text. The profile screen that launches this says "Privacy policy", so there is no expectation that it will have everything in the onboarding flow. And the "terms" page that is displayed has all the information from the summary, just buried in a wall of text. So I really don't think that we need to change the onboarding (that everybody goes through) to make the profile display (that the user has to click on to launch) more comprehensive.

Having said all that, it seems like you should be able to conceptually show both on the profile screen if you wanted to - by just creating separate components that are separate in the onboarding and glommed together in the profile. We don't have to do this, but in the spirit of learning what is feasible, can you clarify what the challenges around this approach would be?

[Abby-Wheelis]

We don't have to do this, but in the spirit of learning what is feasible, can you clarify what the challenges around this approach would be?

We could absolutely still have both in the profile if we wanted to, with exactly the approach you described, and you have a really good point that the information is still there, just woven into that "wall of text". I don't think there would be any significant challenges, as the two components we could "glom" together would just be patches of text, easily embedded into the contents of the popup.

I think I was really just trying to point out that it was a difference between the "old" way and the "new" way and ask if we wanted to preserve the explicit presence of summary in profile. Which as you pointed out, isn't necessarily expected or needed, since many users may never open that popup anyways.

[JGreenlee]

After some more work on #1032, 'summary' and 'consent' are now separate pages. The summary does not show on the Profile tab and is only seen during onboarding.





[JGreenlee]

This was implemented in https://github.com/e-mission/e-mission-phone/pull/1032 and has been on master for a while