Turn the Experiments screen in the Gutenberg plugin into Gutenberg Labs (a home for beta testing)

[critterverse]

There was a great discussion in the Making WordPress Design channel this week around making easier pathways for new contributors to get involved with testing and other ongoing work in core: https://wordpress.slack.com/archives/C02S78ZAL/p1618425099083700

I think there’s an opportunity to add a layer of user-friendliness around beta testing by transforming the “Experiments” screen in the Gutenberg plugin into a cool landing page that introduces the larger experiments (with opt in/out) as well as any blocks in beta (a label explored in #26632).

We could call it Gutenberg Labs 👩‍🔬🔬

(Note: There was some discussion around the idea of "beta" and "labs" in the Full Site Editing Go/No Go demo from earlier this week — around 35min into the video)

Here’s what the Experiments screen looks like currently:

Curious to hear if others have had similar ideas and if there's enough interest to kick off a design explore!

[annezazu]

I would love to see this happen as it would make for a great education tool based on my experience with the FSE outreach program. Great thinking! This feels like a missed opportunity right now.

[paaljoachim]

Hey Channing! That is a very interesting idea! Extending the Gutenberg experimental screen is a really nice idea! Gutenberg Labs. I like the sound of that..:)

[chaion07]

'Gutenberg Labs' Pretty cool idea. Would love to see the concept put into test. +1 from me 🙌

[javierarce]

This is a fantastic idea! I also like that the name evokes a certain combination of the Arts (Gutenberg) and the Sciences (labs).

[carlomanf]

This is a very bizarre conversation for me. If "experimental" is scary, then how is "labs" friendly? A lab is a place where experiments happen. Also, experiments in the plugin are very different from newly merged features in 5.8 and definitely can't have the same name.

[karmatosed]

I love this as an idea and as part of it whilst I am not convinced the naming matters, it is worth consider what it translates to and means across all users. I think the biggest piece not even done now and to do is making it far more appealing and friendly. Why would you want to test right now looking at that page?

[gziolo]

We could call it Gutenberg Labs 👩‍🔬🔬

The only issue I see is that "Guteberg Labs" is different than what would be considered "WordPress Labs". In WordPress core we only include features that are no longer experimental in the plugin. To give an example, Query block is only enabled in the Gutenberg plugin and doesn't exist in WordPress core. During the Full Site Editing Go/No Go demo the discussion was about shipping the Query block in WordPress core and labeling it "Labs".

[critterverse]

Thanks so much for the notes and feedback, everyone! Feels like there's a lot of excitement about revisiting the Experiments screen and this has been a great starting point for opening up the discussion.

The most important aspect of this idea is the added info about what it is you’re testing — I’m hoping this can improve user understanding and generally help make the experiments more approachable.

In terms of naming: “Gutenberg Labs” is TBD but it feels significant to go from a sort of unnamed thing to a proper noun that can be referenced in calls for testing (and hopefully help make this screen into a hub).

To give an example, Query block is only enabled in the Gutenberg plugin and doesn't exist in WordPress core. During the Full Site Editing Go/No Go demo the discussion was about shipping the Query block in WordPress core and labeling it "Labs".

^ Thanks for pointing this out, @gziolo. While these ideas are related, the way that "labs" was referenced in the demo should remain a separate discussion.

In the above example of the Query block, I think it would be great to see it introduced on the Experiments/Labs screen in the plugin to encourage testing before it lands in core — but would love to hear different POVs on this.

Maybe we can start by looking at how we might add an information layer to the existing items on this screen.

[javierarce]

As a reference, I'd like to point out how GitHub handles this. Their experiment section is called Feature Preview, and for each experiment, they show a title, a description, an image, a link to the documentation, and a link to give feedback.

Here's an image of the interface from an old announcement post:

And here's a nice little detail: when you disable a feature, they ask you to give them feedback again.

[critterverse]

This is a great reference, thanks @javierarce! Exciting to see how others are approaching this 🎉

for each experiment, they show a title, description, an image, a link to the documentation, and a link to give feedback.

All of these elements would be great to explore and the callouts for feedback feel like they would be an especially big win for this screen — can imagine providing a link to create a new GitHub issue (possibly based on an experiment-specific template) as well as an invitation to join the ongoing conversation in the Make WordPress Slack.

[karmatosed]

can imagine providing a link to create a new GitHub issue (possibly based on an experiment-specific template) as well as an invitation to join the ongoing conversation in the Make WordPress Slack.

Only point I'd consider is how many users understand Github and how much of a hurdle this is. Perhaps your idea of make would be easier or a blog post to ease someone into a conversation @critterverse? It's easy for us to assume Github is accessible when it's really a huge mountain to climb for many.

[critterverse]

Hi all, checking in with an update on Gutenberg Labs (name TBD).

Here’s a Figma file where @javierarce and I have started collecting some sources of inspiration and early ideas for layout/branding. I’m adding a few images here but we welcome anyone who’s interested to jump into the file and add your own ideas there as well! 😁

Here’s a few of the references we’ve collected:

• Github's Feature Preview
• Gmail “Advanced” tab within Settings (formerly known as Gmail Labs)
• Tumblr Labs (visit tumblr.com/settings/labs while logged in)

Here’s some early look & feel inspiration for the branding aspect of this. There are some open questions about how far we want to take the branding but these are just some fun potential directions to get us thinking (sources are linked to in Figma):

Here are some early wireframes exploring layout. Some of the initial directions we came up with were:

• Traditional landing page like the "About Wordpress" page
• Contained page section with its own sub-navigation for accessing each of the experiments
• Layout based on the idea of individual “cards”

Here’s some further work on a cards-like direction, including a cool idea about including a large header illustration that could potentially be commissioned by different artists/designers:

We shared these preliminary ideas in the monthly Design team Show & Tell zoom call, and got some helpful feedback from that session:

• There was an idea mentioned about using animated GIFs as featured images, however that may result in too much motion and cause accessibility issues
• Instead of GIFs, videos could work
• The card format may look too much like a plugin or theme

Shout out to @annezazu and @melchoyce for their excellent recent blog post, Become an Early Adopter With the Gutenberg Plugin, from which we borrowed some copy and visual inspiration!

Note: We're using the content that's currently on the Experiments screen to explore this idea for now, but consider those as a placeholder for experiments that could live here in the future.

[annezazu]

Just noting that as I dive deeper into a quick iteration of the current experiments page and as the FSE Outreach Program evolves/we think about outreach for phase 3, I am keen to revisit this idea and effort. I dug into figma to mock up a more recent example (ignore the mismatched fonts):

These could then be referenced more intentionally in Developer Blog posts, Gutenberg Release posts, and more. In my mind, we’d need the following to supercharge the efforts for each experiment:

• Clearer naming.
• Short description (1-3 sentences)
• Link to a PR or overview issue about the work itself for folks to read more.
• Solid visual to explain what’s being explored (this could coincide with whatever we share in Gutenberg release posts to prevent redundancy).

For the overall page, I think we could run with these awesome designs shared in the past. Ultimately though, the aim would be to increase feedback loops, particularly around Phase 3 and how that work is currently progressing different than phase 2 where features were concentrated in the Site Editor.

The biggest drawback is likely that this could slow down developers or make it cumbersome to quickly add something to this page but I think it would ultimately be worth it, especially since visual assets are often made already with the Gutenberg release posts and, worst case, could be skipped.

[annezazu]

Quick follow up comment to note the discussion in this related and much more contained issue here on updating the Experiments page: https://github.com/WordPress/gutenberg/issues/55174 In particular, I think it's worth reading through with the lens for how folks might be presented with turning anything on (making it clear it's not for production) and to encourage as much feedback as possible, which needs to take into account the experience of landing in GitHub. Props to @spencerfinnell for talking this through.

[hanneslsm]

Regarding the name: There is also the official "Performance Lab" Plugin from the Performance Team. If Gutenberg introduces the "Gutenberg Lab", we must make clear that this is part of Gutenberg. One could assume "Gutenberg Lab" is a separate plugin.

• Performance Lab Plugin
• Gutenberg Plugin
  • Gutenberg Lab

I am voting for @annezazu's earliest designs and simply naming it "Experiments"