Enhance onboarding experience for Performance Lab

[felixarntz]

1031 focuses on revising the UI of the Performance Lab settings screen to be more intuitive about the features it allows enabling.

Separately, we should think about how we can improve the onboarding experience in particular, specifically about enabling all non-experimental features (see #1045) with one click, or at least have some kind of onboarding where those features are recommended.

There are 3 distinct use-cases to consider here, all of which we need to find a solution for:

1. A new installation of the Performance Lab plugin: The administrator should be proactively informed of the available features and which ones in particular are recommended to activate (all non-experimental ones).
2. An update of the Performance Lab plugin to a version that comes with one or more new features (i.e. new standalone plugins exposed as features): The administrator should be proactively informed of the availability of new features, especially if those features are not experimental.
3. An update of the Performance Lab plugin to a version where one of the features has "graduated" from "experimental" to "non-experimental": Unless the site already has the feature active, the administrator should be proactively informed about the feature now being recommended to activate.

Let's think about what this experience could look like. We may not need a full multi step onboarding wizard for this, as it doesn't really involve multiple steps. But at the same time, we shouldn't settle for just an admin notice or pointer, but rather shoot for a more prominent and more intuitive user experience, with minimal clicks required to get to the recommended features being active.

[westonruter]

As I just mentioned in https://github.com/WordPress/performance/issues/1031#issuecomment-1980042835, we also need to be wary of readonly filesystems. If someone, for example, installs Performance Lab on the Pantheon dev environment and then pushes it to production without going through the onboarding experience on dev, they will then fail to be able to successfully do the onboarding on the live environment because it is a readonly file system.

[felixarntz]

@joemcgill @mukeshpanchal27 I have updated the description to consider the third use-case of features graduating from experimental to non-experimental (also see #1045).

Let's discuss what would be a good starting point in implementing such an onboarding experience. A simple admin notice? An admin pointer plus dedicated UI on top of the PL settings screen? A custom screen purely for the onboarding (akin to a more common onboarding wizard experience where the user's entire focus is on the one step to go through)? Just some ideas to consider.

[felixarntz]

Sharing a few ideas here:

• Taking into consideration that neither of the 3 use-cases outlined in the issue description requires a stepped flow, I think that a dedicated onboarding flow screen is overkill for this.
• To generally make it easier to bulk activate features, we could show a prominent button like "Activate all recommended features" above the feature cards. Clicking it would at once activate all non-experimental features (plugins) that aren't already activated.
• For the initial experience, we already have the existing admin pointer that says: You can now test upcoming WordPress performance features. Open %s to individually toggle the performance features. That in combination with the above button would make the new user scenario more intuitive already.
• For the other two scenarios, I think we can use a combination of admin pointer and potentially a modal that is open as soon as you visit our screen:
  • When updating to a version with one or more recommended feature cards (could be an entirely new recommended feature or a formerly experimental feature), the update routine should set an option in the database for which features are the "new" ones.
  • If that option is not empty and the plugins for those new features aren't already active (which is probably always the case), another admin pointer should be shown that indicates that new features are available, pointing to our screen.
  • This option would lead to the cards of the new features, as part of our regular UI, showing some annotation like **NEW**.
  • When landing on our screen in that scenario, a modal is open from the very beginning (somewhat like the "Choose a pattern" modal in the block editor, see below screenshot) which specifically calls out the new recommended features including their description. It would also include a button to activate those new features at once. Alternatively the admin could just dismiss the modal.
    • It may be sufficient to use the same option that stores the features to include a dismissed boolean key, which, when true would lead to that modal being no longer present. This way we can keep showing other administrators the admin pointer at least, and they would still see the **NEW** annotations on the cards when they land on our sscreen.
    • When all of the new features are active, the entire option in the DB would be deleted as at that point it would be pointless and good to clean up. From an end user experience, this would simply lead to the features to no longer be called out as **NEW**, and no more admin pointers about them.
  • The other use-case would be an update which has no new recommended features, but new experimental features. The approach for that could be similar, however with different wording which would be more informative and less of a CTA - simply because those features are experimental, and we wouldn't want to be too "aggressive" in prompting the user to enable them.

Curious about your thoughts @joemcgill @mukeshpanchal27 @swissspidy!

[swissspidy]

A "Activate all recommended features" button of some sorts makes sense. Makes it easier to get started.

I don‘t see us need a „Choose“ modal though. Highlighting new & recommended features on the settings page would already be very helpful.

[felixarntz]

@swissspidy Makes sense. We could also go without any of the modal idea and instead just highlight the **NEW** things. If we do that, we'd only need to include another heuristic or routine to at some point mark those things as "not new", so that they don't remain marked as new until any other new feature is added and would override them.

Potentially we could use a transient with expiration of a number of days where we could assume hopefully at least 1 admin has seen the new features by then.

[westonruter]

Related to this is Core-60992. In versions of WP prior to 6.5, plugins could redirect to an onboarding screen after activating a plugin. This broke in 6.5 with Ajax-activation of plugins. There may be an opportunity for us to hook into whatever solution is come up with to direct the user to the Performance screen to take the next step to activate features after activating Performance Lab. Otherwise, they would only find out about the Performance screen by (1) reloading the plugins screen and discover the Settings link, or (2) go to the dashboard and discover the admin pointer.

[joemcgill]

There seems to be a lot in flux right now with how onboarding flows for plugins in WP should work (see: Core-61040 for continued discussion). I think it best that we defer the question about an onboarding experience after initial activation of the Performance Lab plugin, and instead focus on the messaging/experience we can give users to automatically activate all non-experimental plugins from the Settings > Performance screen.

We already show the following admin pointer after activation.

Once someone gets to that screen there should be a single CTA that encourages folks to install all recommended features. Here's a rough mockup (copy is just for example)

[westonruter]

@joemcgill This sounds great. How about putting the "Active All" button in the spot where the "Add New Post" button is on the edit posts screen?

I think we can then avoid having to add copy, or else that copy could be added to an admin pointer that is specifically connected to that "Activate All" button.

[joemcgill]

I like the positioning of the "Activate All" button. I still think that this page is too sparse and we are missing the opportunity to provide some context about why all these "cards" are on the screen, and what the value is for people to activate them.

It's also not clear that the "Activate All" button would only activate non-experimental features unless we separate the experimental cards into a separate section or provide some other context.

[westonruter]

Yeah, it does feel a little sparse.

As for the button, I suppose there could be two after the heading:

• Activate Stable
• Activate Experimental

Once they are all active then the button would go away. I'm not in love with this though. Still, that button position is familiar for users in WP.

Perhaps the Privacy screen has some design patterns we can reuse, with tabs (e.g. Stable and Experimental), followed by copy, followed by controls:

That said, I've probably been to the Privacy screen only a handful of times, with this time to take the screenshot being the 5th 😄

[felixarntz]

@westonruter @joemcgill I think having the button in the familiar location is a good starting point, in principle. I am a bit concerned about the wording though.

• "Activate all" is not quite what it does.
• "Activate non-experimental" is too verbose IMO.
• "Activate stable" is IMO the best of the 3 options, but still it uses language that IMO assumes software version familiarity (stable, beta, RC).

I think "experimental" is easy to understand, so "non-experimental" is sort of easy to understand as well, but maybe there's a better option. Initially I was thinking to use something like "recommended".

Alternatively, we could use a different UI where the button appears alongside a brief description, like what @joemcgill proposed above. Maybe something like this:

• A text like: Improve the performance of your site by activating the following features. All non-experimental features are recommended for use. Alternatively, you can activate all features (including experimental) or individually activate features.
• A primary button (i.e. the blue background) "Activate recommended", which activates all non-experimental features.
• A secondary button (i.e. like the other buttons on the page) "Activate all", which activates all features.

I would in any case also recommend to keep this whole UI conditional. I'd say that, if all non-experimental features are active, it shouldn't be shown, as it would provide little to no value then. The only value would be to bulk-activate the experimental features then, but since we wouldn't recommend that and the user initially chose not to do that, I think it's not worth keeping this UI.

WDYT?

[westonruter]

Yeah, I wasn't sold on my wording either. I like the idea of having two buttons of different colors, one being the primary blue activating all non-experimental features and a secondary button that activates everything. These could even be inline with the heading if we don't go with additional copy (and perhaps the copy could show up as a tooltip or admin pointer?). Or the buttons could be presented in an info admin notice with copy, where I think the notice styling would bring eyes to it without being obtrusive.

I hesitate about "recommended" because that could imply that experimental features are not recommended, but we do want users to activate them. What if we instead position experimental plugins as "beta" or "early access"?

[westonruter]

Punting from 3.1.0 to n.e.x.t (3.2.0). Normally this would be on June 17th, but given that WCEU is immediately prior to this, I think it makes sense to do a mid-cycle release, perhaps June 10th (or even the week prior).

[felixarntz]

@westonruter

I hesitate about "recommended" because that could imply that experimental features are not recommended, but we do want users to activate them.

I'm not sure it implies that. "Recommending" to activate these plugins would effectively be "encouraging" to activate them. Not encouraging something doesn't mean it is discouraged. It's just not encouraged, or at least less strongly.

What if we instead position experimental plugins as "beta" or "early access"?

I don't think the term "experimental" needs to be changed at all, IMO it's clear as is. The primary problem is that we need an easy-to-understand term for all of the non-experimental plugins so that we can refer to it on the primary CTA button. Changing the term "experimental" doesn't necessarily help with that.

[joemcgill]

I think @felixarntz's initial proposal above for button text is good as I understood it.

• Primary button (i.e. the blue background) "Activate stable", which activates all non-experimental features.
• Secondary button (i.e. like the other buttons on the page) "Activate all", which activates all features.

Here's my pass at a description:

"Activate the following features to improve the performance of your site. Stable features have been tested for use in production environments. Experimental features are new features that are still in development, which are ready for early testing and feedback."

[westonruter]

Related to this is Core-60992. In versions of WP prior to 6.5, plugins could redirect to an onboarding screen after activating a plugin. This broke in 6.5 with Ajax-activation of plugins. There may be an opportunity for us to hook into whatever solution is come up with to direct the user to the Performance screen to take the next step to activate features after activating Performance Lab. Otherwise, they would only find out about the Performance screen by (1) reloading the plugins screen and discover the Settings link, or (2) go to the dashboard and discover the admin pointer.

As part of WP 6.5.4 it seems the current direction is to allow a plugin to add a filter that will cause activation to redirect the user to an onboarding screen. See Core-61269. This would make sense to do as part of the Performance Lab plugin especially, to redirect users to the Performance screen upon activation, since activating the Performance Lab plugin on its own does not include any of the features (other than Site Health and Server Timing).

[westonruter]

As part of WP 6.5.4 it seems the current direction is to allow a plugin to add a filter that will cause activation to redirect the user to an onboarding screen.

This got punted to 6.6. For the minor release, the Ajax activation was reverted. See Core-61040.

[joemcgill]

We got a recent bug report (#1292) from a user who expected to be able to deactivate the feature plugins from the performance settings screen after installing them. While this is expected behavior and not really a bug, the UX is not the most clear and is something we could probably improve as part of the onboarding process improvements.

[westonruter]

See also https://github.com/WordPress/performance/issues/1031#issuecomment-1989073350 and the surrounding comments regarding whether feature deactivation should be available from the Performance screen. (I think it should).

[westonruter]

Moving this to On Hold as this should be worked on once Core-61040 settles.

[felixarntz]

@westonruter Not sure we should block this on the Trac ticket. Plugins have already provided onboarding experiences regardless of core support, and it's unclear whether that Trac ticket will move forward in the foreseeable future. It's "Awaiting Review", and the last comment was 2 months ago, so I think we need to decouple our work from it.

This issue is mostly about having a UI that facilitates onboarding upon installation or update (e.g. a new screen, or a variation of the current PL features screen). How to get there is what would vary depending on how core may implement a plugin onboarding API, but for now we could use what's already possible, e.g. a redirect or an admin pointer. So if we build the functionality into our own screen, I don't anticipate major changes being required to support whichever way core provides in order to get users to that screen.

[westonruter]

Yeah, I thought there was more activity on the ticket as it was somewhat of a hot topic during the last release. But it does seem to have quieted down. I was hoping we could build on top of what was being proposed for core as opposed to reinventing the wheel, as many plugins currently do. Or in other words, to take a blessed approach for however core wants plugins to do onboarding. In any case, as you say, it probably won't require much work to be compatible with whatever core ends up doing, whenever that does happen. Moving this out of On Hold.

[felixarntz]

See https://wordpress.slack.com/archives/C02KGN5K076/p1724773677843549 for Slack discussion about potentially doing user research around this at WordCamp US this September.

[felixarntz]

Sharing here a summary of the responses to the onboarding feedback form: https://docs.google.com/presentation/d/1kDHYXe7IpQSN5aX9rFYIeEzklyPX7XoD2XKw1_vhkRY/edit

My summary of the takeaways here:

• The biggest issue at the moment is that (at least some of) the feature descriptions are not clear enough.
  • This is particularly important to address as the majority of folks that responded read the individual feature descriptions rather than immediately activating a bunch of them just because they are part of the plugin.
• In some instances, it's not clear that the features are individual plugins. Reusing the plugin card UI alone is not enough, so we need to clarify that more explicitly in the UI.
• The current "(experimental)" markers for experimental features work well enough. There was one complaint, so we may want to improve that at some point, but it shouldn't be a priority.
• Finding the features screen is not a relevant problem.
• Bulk-activating features is not a problem. However, there may be a bug when activating features too quickly after each other, which we should address.

The last point is probably the most urgent one to focus on, as it seems there's something broken that several users ran into. Other than that, we need to revise some of the feature descriptions. Coming up with better copy may take some time though.

Curious to hear anyone else's interpretation of these responses.

[westonruter]

• Bulk-activating features is not a problem. However, there may be a bug when activating features too quickly after each other, which we should address.

It would be very nice if activating a feature would use Ajax instead of doing full page refreshes.

[joemcgill]

Reading through the feedback, it looks like there were several positive takeaways:

• 100% of respondents were able to locate the setting screen (mostly from the admin pointer) after activating
• Nearly all of respondents (9/11) successfully activated features, with a majority (6/11) selecting which to activate based on the descriptions
• It is not clear to users that they are installing a plugin when activating features, but I'm not so sure that's a major problem
• 10/11 respondents could successfully identify that some features are experimental
• The most common frustration was not understanding what some features would do. I think this is similar to your observation, @felixarntz, and improving the descriptions for each feature could help improve those numbers.

I'd also caution putting too much weight on this survey, since it's a pretty small sample size, and it's unclear whether these responses are biased towards technical users who were likely to see the link shared on Twitter/X.

[felixarntz]

Thanks for sharing your observations @joemcgill. They make sense to me, just one additional reply:

It is not clear to users that they are installing a plugin when activating features, but I'm not so sure that's a major problem

It's worth noting that this is a concern from a plugin directory compliance perspective, as any plugin installing other plugins should clearly indicate so. Since we now know replicating the UI was not sufficient to make that clear, we may need to put more explicit wording in place.