search

WordPress / gutenberg

The Block Editor project for WordPress and beyond. Plugin is available from the official repository.
https://wordpress.org/gutenberg/
Other
10.32k stars 4.12k forks source link

Consider real-world usage of WordPress to quantify assumptions made across the Gutenberg project #6228

Closed johnbillion closed 6 years ago

johnbillion commented 6 years ago

Issue Overview

The REST API project was a big focus from 2012 up until the completion of its inclusion in WordPress 4.7. Its development spanned four years.

Despite this lengthy period of development, it wasn't until Gutenberg's development started in 2017 that shortcomings of the REST API were really seen on an impactful scale. The Core REST API Task label is tracking issues in the core REST API that have been identified as a result of what the Gutenberg developers are trying to do with it.

All that is to say that the REST API went through four years of development yet has shortcomings that were only identified when developers started building an actual end-user facing product that consumes it. Attempts to replace admin-ajax.php functionality in the WordPress admin area with REST API calls have also hit problems.

What's my point? My point is that the REST API was not exposed to enough real world usage during its development cycle, and this is only now becoming apparent when developers start building innovative applications on it and shortcomings are discovered. Not enough people built things that used the REST API during its development in order to discover whether what was being built actually solved problems.

This ticket will serve to help avoid the same problem happening to the Gutenberg editor. Before a merge proposal can even be considered, a substantial amount of real world usage of Gutenberg needs to be performed, gathered, analysed, iterated upon, and considered as guiding aspects of its continued development.

Real-world usage of Gutenberg covers a much wider spectrum than the REST API. It covers:

I've opened this issue because I believe that the project in its current phase isn't considering real world usage, and I don't believe enough consideration has been put into what problems it's solving. Two open issues illustrate this point:

The result is that the project is making many assumptions about end users, and how they'll use a Gutenberg powered WordPress site. Some user testing has been done (most recently I believe at WCUS in December) but with a narrow and controlled scenario that isn't representative of the way that 30% of the web uses WordPress. I currently enjoy using and experimenting with Gutenberg, but I'm not remotely representative of the target user.

In order to succeed, the project needs to step back somewhat and consider what it's trying to accomplish, for whom, and in what capacity.

I want Gutenberg to succeed. In fact, I think Gutenberg has to succeed, but it won't unless real world usage is moved to the forefront of considerations taken into account during the development of the project.

To that end, a question: How can real-world usage considerations be embedded into the project's ongoing design and development to ensure that it is a success?

Addendum: My own availability to work on this issue, and Gutenberg itself, is unfortunately almost nil (ironically due to working on a Gutenberg-powered project for a client) but I will keep an eye on this issue as and when I can.

Edit: I updated the issue title for clarification.

glueckpress commented 6 years ago

I think Gutenberg has to succeed

What is success? Gutenberg will have succeeded when … ?

grahamarmfield commented 6 years ago

I believe also that some informal accessibility related user testing was done at CSUN recently.

karmatosed commented 6 years ago

Thanks for lending your voice to this issue, @johnbillion.

As you and many others know, usability testing is one of my favourite topics. We've had small, curated test groups so far because I believe it is important to be aware of who and what we've tested. My hope like most is that as Gutenberg draws closer to merge, we'll see even more tests.

Can we ever do too much testing and get too much feedback? Probably not. Adding brand new functionality to 30% of the web isn't without risks. We've iterated a lot on the existing features, so much so it is now time to get a stable product, to then reach out to more and more diverse testing groups.

I'd love to hear more about what you, John, and others are building with Gutenberg. What we can do to make that process easier for those creating and those using the experience? What are the stress case stories we need to hear? The more people use, build on, and launch with Gutenberg, the closer we'll get to a product that's ready for 30% of the internet.

danielbachhuber commented 6 years ago

@johnbillion I'm actively working on Gutenberg now if there are any specific conversations you'd like to pull me into.

0aveRyan commented 6 years ago

I would first like to applaud hard work, deliberate attempts at inclusion and commendable patience the core Gutenberg team has shown since inception.

Often for better, Gutenberg is opinionated. It takes realities of data storage, legacy content and user needs into consideration. But I think my concern is the opinionated nature reproaches some historical control from users (and devs), and that risks alienating some markets. It could also open new doors for WordPress that sustain it ¯_(ツ)_/¯.

Metaboxes encouraged users to show-and-hide, rearrange and create many different experiences. This is a paramount strength and weakness.

Many meta boxes can be open, however, a single tab/accordion can be open. Gutenberg's reduction is perhaps powerful, perhaps healthy, perhaps enables more complex products going forward, but does reduce the HUD ability of scrolling a long page without interaction and may limit some use cases.

I think my strongest concerns are:

chrisvanpatten commented 6 years ago

My team and I are just kicking off a project to rebuild a WordPress site for a semi-prominent/well-circulated North American magazine. We're planning to be Gutenberg-first, and are also aiming to use it for managing the homepage layout (in addition to article and static page content). Target launch is late June/early July.

I'm hoping to publish blog posts along the way about the experience, and will definitely be encouraging the team to contribute back to Gutenberg, especially as we work with the magazine's editorial team and get a sense of how they use it in their workflow. I agree that seeing Gutenberg in real world situations is critical and hope our experience can be useful in helping it develop.

If there are any other specific ways we can help the Gutenberg team, we'd definitely love to do so!

davisshaver commented 6 years ago

Great note and admirable articulation of a sensitive topic. @johnbillion has identified a number of user personas here who could be negatively impacted by the Gutenberg release. Many of them are users who I'd characterize as .org, vs .com. Some of the users can be characterized as enterprise, vs consumer.

I think it's a fair statement that the .com + consumer has been a key demographic focus for the Gutenberg team.

Many of the developers in this thread represent .org + enterprise. An admirable example is set by @chrisvanpatten and other teams building now for Gutenberg. I see these as critical efforts to understanding Gutenberg and any delta between its current state and what's needed for a successful launch.

But regarding these .org + enterprise efforts I am curious about the timing. Would any of these projects launch before Gutenberg was merged into Core? Is any multi-author, high-capacity newsroom planning to use Gutenberg before a 5.0 release? Anecdotally I've myself been discouraged from doing this at present as the risk/reward isn't there.

Matt and Automattic have done an okay job at communicating why Gutenberg should be a priority for WordPress, and as project founder he has the BDFL's right to set strategy in a non-democratic manner. But to hedge against a potential unsuccessful launch of Gutenerg I think A8c could help the community by encouraging, assisting, and subsidizing enterprise-level tests prior to 5.0. While public mentions of a release date have been sparse there does still seem to be a rumor that May/June may see 5.0. This timeline seems rushed, and as I pointed out on Twitter, Gutenberg seems to be a misfit in the "deadlines aren't arbitrary" regard already. So delaying further seems fine.

chrisvanpatten commented 6 years ago

@davisshaver I can only speak for ourselves, but we are absolutely planning to ship regardless of whether Gutenberg has been merged (and are operating under the assumption that it won't have been merged yet).

Also worth noting that Automattic is encouraging and assisting enterprise deployments, through WordPress VIP. At the recent #BigWP event in New York, they announced that they'll be publishing some Gutenberg training to the public (potentially tomorrow?) and releasing a more configurable version of the Classic Editor plugin (sometime in June or so).

We aren't a VIP partner so it's nice to see them making (some of) these resources available to the public, and I hope they continue to do so!

davisshaver commented 6 years ago

Agreed! The engineering resources are good and appreciated! My point regarding Automattic was not about documentation or code quality. It's more a general concern about unknown unknowns and how A8c has a special responsibility having mixed its corporate and open source activities. As very well demonstrated by the VIP resources ;).

What's not documented–what you will be generating–is the learning from launch in a high volume newsroom. To my knowledge, if you were to launch, you would be among the first organizations to meet the .org + enterprise designation using the product. This seems suboptimal to me; more usage would be better, and all the user types @johnbillion mentioned would ideally get their time to road test the experience.

At present it seems the plan is to kick Gutenberg out the door & expect that enterprise newsrooms would switch sometime late this year. But what if we discover something critical during that first period? Before enterprise has migrated, after it's in Core. That period where users are forming opinions about upgrade path and Gutenberg. We only have a single chance to get that right. We all want to prevent any rift in the community or needing to add a LTS version. Personally I don't feel the de-risking to date has been adequate for the scale of the WordPress project.

jasonbahl commented 6 years ago

I'm working on a site at 10up for a government division and we are planning on shipping with Gutenberg as well, regardless of whether it's merged or still a plugin. Timeline to launch is also June/July.

Like @chrisvanpatten our initial plan was to really go all-in and use Gutenberg for pages, posts, and even templates like the homepage or other landing pages that would commonly be more or less hard-coded archive-*.php templates with widget areas or customizer controls or whatever.

However, some issues we're running into in using Gutenberg that are essentially "blockers" (pun intended) for going all-in:

I have other general concerns, but these are some of the ones that are really hindering current real world use for us.

danielbachhuber commented 6 years ago

Awesome write-up, @jasonbahl 💯

ZebulanStanphill commented 6 years ago

I am currently using Gutenberg for my blog posts, but I do not intend to use it for any pages on websites yet, mainly due to the lack of responsive, non-equal width, and variable width columns, as well as the lack of sections. Without those, I simply can not use Gutenberg for serious page building. I understand that they are coming in the future, but I would really prefer if they came before the WordPress 5.0 release.

mrleemon commented 6 years ago

I've tested 2.7 recently and one of the main reasons of using Gutenberg, the columns block, continues to be really difficult to work with, so at the moment using Gutenberg on client websites is a no-go for me.

karmatosed commented 6 years ago

Thank you everyone so far for the awesome feedback. I want to step specifically into answering points if I can to try and make sure we both have issues and more information for everything.

@chrisvanpatten really excited to see your launch!

If there are any other specific ways we can help the Gutenberg team, we'd definitely love to do so!

Yes so many ways! We always need code contributions for example PRs and triage to the issues. However, beyond that using and feeding back where the stress points are is an invaluable contribution. Publishing that post you are saying you are going to do is also a huge contribution, thanks.

@davisshaver if I may just reflect on your comment here:

I think it's a fair statement that the .com + consumer has been a key demographic focus for the Gutenberg team.

Actually the most source of feedback has been from the .org + enterprise. I know that may come as a surprise to many, but it's true. I note feedback as the product is built and influenced by the feedback it gets.

Just to ensure everyone has the information, as has been mentioned I would point to https://testgutenberg.com/ and https://vipgutenberg.com/ for some awesome resources. There are also other courses like the awesome ones @zgordon runs: https://gutenberg.courses/.

@jasonbahl this is exciting to read!

I'm working on a site at 10up for a government division and we are planning on shipping with Gutenberg as well, regardless of whether it's merged or still a plugin. Timeline to launch is also June/July.

Let me try and address each point you raise:

the lack of good page-template support.

Can I check what you mean there and if the following issue helps or not?

If that doesn't, can you maybe phrase exactly what is blocking and we can get an issue made for you, or feel free to make one of course.

the lack of ability to whitelist/blacklist both for the entire editor

I don't see an issue for this specifically so wondering if you'd like to make one? I do know of plugins that limit like https://wordpress.org/plugins/block-options/. I would like to dig a bit more there though.

the lack of a Server Side API to provide validation on user input.

Does this cover that: https://github.com/WordPress/gutenberg/pull/5602? I just want to make sure we have an issue relating to this.

I have other general concerns, but these are some of the ones that are really hindering current real world use for us.

I would love to hear those either here or please reach out to me on chat.wordpress.org.

davisshaver commented 6 years ago

@davisshaver if I may just reflect on your comment here:

I think it's a fair statement that the .com + consumer has been a key demographic focus for the Gutenberg team.

Actually the most source of feedback has been from the .org + enterprise. I know that may come as a surprise to many, but it's true. I note feedback as the product is built and influenced by the feedback it gets.

To clarify this, I mean users inside .org + enterprise environments, not the engineers serving them. @karmatosed has there been a test inside a high volume site yet?

greatislander commented 6 years ago

Just to ensure everyone has the information, as has been mentioned I would point to testgutenberg.com and vipgutenberg.com for some awesome resources.

@karmatosed vipgutenberg.com is only available to WordPress VIP customers, correct? That leaves many (most?) developers and agencies out.

zgordon commented 6 years ago

@greatislander two of those VIP courses are white labeled versions of what is available at gutenberg.courses and it is for VIP clients only, correct. You can access the same content at gutenberg.courses tho :)

They are also launching some free content on vipgutenberg.com today as well.

greatislander commented 6 years ago

Thanks for the clarification @zgordon!

chrisvanpatten commented 6 years ago

@davisshaver I don't want to name names (not my place!) but I can say that there are definitely organisations experimenting with Gutenberg in editorial. I don't know of any larger org using it in production, but it's definitely being exposed to non-developer users in enterprise.

davisshaver commented 6 years ago

@chrisvanpatten Right, as you noted, this was discussed at the recent Big WP meetup. But my point is that ideally > 0 orgs have tested in production before 5.0 release. I am currently making plans to launch OnwardState.com on Gutenberg next week.

karmatosed commented 6 years ago

To clarify this, I mean users inside .org + enterprise environments, not the engineers serving them. @karmatosed has there been a test inside a high volume site yet?

@davisshaver, there has been some feedback but we can always have more. It also depends on what we're talking about with high volume. It's important to get all stress cases I absolutely agree.

That's great you are launching on next week. Anything you can pass on as a lesson there in maybe a blog post? If unable to post public please reach out private. I would love to learn more from everyone as to their experiences.

@greatislander: apologies, I should have added that it was coming soon. As @zacgordon says it's all going to have free content as of today - which rocks!

Thanks all for the continued tone and quality in this issue, it's incredibly important to get insights like this.

jasonbahl commented 6 years ago

@karmatosed I'll elaborate a bit more on my previous comment.

the lack of good page-template support.

Currently, when a user selects a Page Template it's trivial to add contextual awareness to metaboxes so that the user can enter content specific for that template. If they switch templates, they will see different metaboxes.

Ideally, when a Page Template is selected, a user would be presented with a "Block Template" that allows them to edit the page template.

@melchoyce presented a compelling vision for this at LoopConf, but that was just a vision, not current functionality of Gutenberg.

There are issues related to this. In fact, searching the Github repo for "page template" shows 136 results currently: https://github.com/WordPress/gutenberg/search?q=page+template&type=Issues

Some relevant ones: #3588, #5521 #5537 #4482 (probably more out there that are relevant too)

the lack of ability to whitelist/blacklist both for the entire editor (nobody needs 62 blocks out of the box), but also for white/blacklisting allowed blocks that can go into nested blocks. . .(like what blocks should be allowed in a 1/8 column or other specific nestable blocks etc) – and other conditions, like user role, page template, post_type, etc.

There are documented ways to whitelist/blacklist, but they're currently not functional. I've elaborated more in this issue: #4848, which is a duplicate of #5895 and in both cases the solution presented is moving Block Registration to the Server, a'la #2751, where I've also written a few books worth of my thoughts :wink:

Which leads us to the next point:

the lack of a Server Side API to provide validation on user input. Client validation is nice, but we think it's still wise to never trust the client

I've elaborated quite a bit on why I think the Server Side API needs more attention in #2751. Also engaged in a discussion about the importance of Gutenberg having a Server Side API on make.wordpress.org in regards to the Native iOS and Android Apps plan to support Gutenberg: https://make.wordpress.org/mobile/2018/03/21/gutenberg-on-mobile/#comment-19383

The project we're working on now expressed very early on that they want to have some content be editable by certain roles and locked to other roles.

Blocks are a great example of how this could be possible, and there are probably hacky ways to try and do this on the client at some level, but in reality the server needs to have the final say on what users can do with what blocks. The client shouldn't be trusted. The client should provide validation to it's best ability, but the server needs to have the final say.

Being able to validate input from the client is pretty critical for anyone to really adopt Gutenberg I would think.

There's been prototypes of Collaborative editing on blocks as well. . .without a good Server Side API to know what data can/cannot be a part of a block to act as that intermediate contract negotiator between clients, you're asking for folks to get hacked. If we're talking directly client-to-client, I imagine someone will figure out quite easily how to send something that starts tracking your keystrokes or whatever else. I'm not a hacker, I don't know what a hacker would come up with, but I can tell you it would not be fun to deal with. Having a server do the proper validation on Block input before sending data across the wire to another client is pretty crucial when the day comes to have collaborative editing.

FWIW, it does look like there's action on this here: https://github.com/WordPress/gutenberg/pull/5802

But, all the resources, documentation, etc are still telling everyone how to do everything on the client. There needs to be a BIG push to get this Server API solidified sooner than later then go through and update all materials out there, including third-party materials like the courses @zgordon and the like have put together on how to properly register blocks via a Server API and letting folks know that client-only registration of blocks is generally not recommended.

I have other general concerns, but these are some of the ones that are really hindering current real world use for us.

karmatosed commented 6 years ago

@jasonbahl thanks for those replies. I totally agree that Mel's vision is exciting and it will happen - but it's not happening now. I wanted to say thanks for diving into the repo, so good to see you appearing on a few issues.

If you ever do want to point out other concerns or issues please reach out either here or via chat.wordpress.org Slack also. It's important as a team we hear them from everyone. That extends to everyone in this discussion.

jasonbahl commented 6 years ago

@karmatosed

it will happen - but it's not happening now

Any clarification on what this means?

What part of it isn't happening? Good support for nested blocks? Support for controlling what blocks can be nested within other blocks? The ability to swap block templates on the fly when "Page Template" is changed?

Would be good to have clarity on what current features, such as nested blocks, are getting finessed so they work well or removed because they're too hard to get working or whatever.

melchoyce commented 6 years ago

What part of it isn't happening? Good support for nested blocks? Support for controlling what blocks can be nested within other blocks?

Hey, good questions. Let me take a stab at answering, based on where I think we're at right now.

Good support for nested blocks? Support for controlling what blocks can be nested within other blocks?

As far as I can tell, these improvements are ongoing: https://github.com/WordPress/gutenberg/issues?q=nested+label%3A%22%5BComponent%5D+Nested+%2F+Inner+Blocks%22

I think we're going to finish up a lot of the bones supporting page layouts for 5.0, but we're not going to see native Gutenberg page layouts until after the initial release. The work I've been doing on being able to select page layouts (in lieu of the old page templates) is still in design, and won't be making it into the initial release of Gutenberg.

The ability to swap block templates on the fly when "Page Template" is changed?

This is definitely planned, and I've been working on some mockups around this, but I don't see it happening until after 5.0.

I would personally like the second Gutenberg release to focus on pages — page layouts, better blocks for creating complex layouts, more dynamic blocks, etc. — but that's still TBD, based on how the initial release of Gutenberg goes.

m commented 6 years ago

@johnbillion, totally agree with everything up to the last section of your ticket. 💯

To use the terminology in an earlier comment, .org and enterprise feedback is coming in at a healthy clip right now, and has been influential and great. I am actually more concerned about some of the personas talked about that are more regular end-users who either had someone else set up their site or are more .com.

What I think will solve that is (1) in-dashboard promos for the Gutenberg plugin to be installed on .org sites, put in a future 4.9.x release and (2) availability of Gutenberg on .com, likely also opt-in at first, in both wp-admin and Calypso. I believe this will increase our active sites and testers by an order of magnitude, from the ~10k sites we have today into the hundreds of thousands, which should dwarf any core feature we've merged in recent memory. Part of my uncertainty about release date is purely a function of how much blocking feedback comes from exposure to those new audiences. (It's possible that the great user testing we've been doing so far will mean it's actually not that bad.)

I think we've learned a lot of lessons from the REST API merge, and I'm glad you raised the issue.

karmatosed commented 6 years ago

@jasonbahl, thanks for asking for clarification and @melchoyce thanks for adding your input there, phase two is going to be exciting after laying the groundwork.

danielbachhuber commented 6 years ago

@johnbillion Given the timeline proposed at WCEU, are there any specific, actionable steps you'd suggest we take that aren't already underway?

johnbillion commented 6 years ago

I'm really pleased to hear there's a plan in place to begin user testing of Gutenberg on WordPress.com. It's probably a good idea to open an issue that outlines the plans that are in place for gathering and analysing the feedback that will be submitted by users as part of this testing phase so that it can be tracked, triaged, and actioned by the community. @danielbachhuber Is this something you can coordinate? Unfortunately I'm still short on time.

If I had to pick from my original list above, I think it's important to look at the impact that Gutenberg has on the sort of sites that make up the majority of the long tail of WordPress sites but which are (ironically) generally seen less frequently by the core WordPress community: sites that are cobbled together with a large number of plugins including page builders, e-commerce plugins, photography galleries, membership plugins, multilingual plugins, and off-the-shelf themes. There is a real risk of widespread damage to the reputation of WordPress if the Gutenberg editor breaks these sorts of sites en masse or confuses the editing experience.

glueckpress commented 6 years ago

I’d plus 💯 that last paragraph from @johnbillion if GitHub allowed me to. Unfortunately, that particular, wide-spread type of site could be hard to get a hold of for testing.

For example, plugin customer support allows me to see potentially problematic back-ends on an almost daily basis, and that’s probably true for some other support folk, but obviously that doesn’t enable or allow us to test Gutenberg on those sites.

What support reps from plugin providers, myself included, could help doing is spread the word. In my case, our product doesn’t even interact with Gutenberg directly, and that might be the same for quite a few other plugin providers. However, that doesn’t mean we should remain silent. If Gutenberg turned out to blow up our customers’ websites, guess where those customers come first for help…

I’m going discuss ways of making WP Media’s support customers aware of Gutenberg with our C level, and I’d like to encourage other support reps from other companies to do the same. (Although I’m aware there’s a good chance I’m just late to the party and a lot of that is already happening.)

mrleemon commented 6 years ago

I think it's important to look at the impact that Gutenberg has on the sort of sites that make up the majority of the long tail of WordPress sites but which are (ironically) generally seen less frequently by the core WordPress community: sites that are cobbled together with a large number of plugins including page builders, e-commerce plugins, photography galleries, membership plugins, multilingual plugins, and off-the-shelf themes.

This!

danielbachhuber commented 6 years ago

It's probably a good idea to open an issue that outlines the plans that are in place for gathering and analysing the feedback that will be submitted by users as part of this testing phase so that it can be tracked, triaged, and actioned by the community. @danielbachhuber Is this something you can coordinate?

Yep, certainly, if it's appropriate / effective.

One project to put on everyone's radar is the #hosting-community's work on Try Gutenberg Hosting Support Best Practices. Essentially, the goal is to identify best practices for hosting support teams as it relates to Gutenberg being made available to a wider audience. Most specifically, we want support teams to:

  1. Understand what Gutenberg is and how it works.
  2. Be confident in diagnosing support requests / bug reports.
  3. Know how to act on the result of their diagnosis (let it be opening a new Gutenberg issue, reporting the conflict to the plugin author, etc.).

We've been discussing this document in our weekly chats for a month or so now. Our original plan was to "test run" the document with DreamHost and InMotion support teams before advertising more broadly. However, it might be worthwhile to publish a block post about it sooner.

cc @jadonn @getsource @0aveRyan

If I had to pick from my original list above, I think it's important to look at the impact that Gutenberg has on the sort of sites that make up the majority of the long tail of WordPress sites but which are (ironically) generally seen less frequently by the core WordPress community: sites that are cobbled together with a large number of plugins including page builders, e-commerce plugins, photography galleries, membership plugins, multilingual plugins, and off-the-shelf themes. There is a real risk of widespread damage to the reputation of WordPress if the Gutenberg editor breaks these sorts of sites en masse or confuses the editing experience.

Yes, and this is certainly something that the #hosting-community has good visibility into and is actionable to get in front of. A little bit of effort from some small teams proactively testing on customer staging sites could go a long way.

tofumatt commented 6 years ago

While the discussion in this issue has surfaced many good points, I think a good rule for our issues is they should be actionable and discrete/specific. This issue is quite broad in its scope and I'm not sure how to measure it "completed".

I hope that a lot of the good points around research and testing are internalised by the team working on Gutenberg and on other parts of WordPress. I'm going to close this issue as I don't think it's actionable as-is, but if there are discrete issues to come out of it that can be filed separately, that's cool 👍