Gutenberg as a framework: streamline the experience

[youknowriad]

Gutenberg can be used as a platform/framework to build block editors. Mainly thanks to the @wordpress/block-editor package. That said, the experience today is not as straightforward as it can be. There can be a lot of small gotchas and hacks you need to do in order to achieve the desired result. This issue is going to serve as an overview issue to collect and fix all these issues.

• [ ] Peer dependencies: Installing the @wordpress/* packages using npm shows a number of warnings due to unmet peer dependencies, with the latest updates to npm, we should make sure we're using peer dependencies properly. For instance it seems that React shouldn't be a peer dependency for us but instead a real dependency of @wordpress/element since this package is supposed to be used as a replacement for React.
• [x] We use process.env.something in a lot of places in Gutenberg codebase. All JS environments do not provide these, for instance Vite relies on import.meta.env instead. We need to at least not break the JS execution if process is undefined. Assess whether import.meta.env is a better alternative these days.
• [ ] There's some weird focus styles by default in the RichText elements, Maybe some reset CSS need to be bundled by default in the @wordpress/block-editor stylesheet.
• [x] Render a <BlockEditorProvider><BlockCanvas /></BlockEditorProvider> should be enough to render the most basic block editor. Today, we're forced to render a number of wrappers and providers or else the experience is broken. Find a way to bundle these providers or make them optional (bundle them if the provider hasn't been rendered already)
  • [x] <BlockEditorKeyboardShortcuts.Register /> #53910
  • [x] <WritingFlow> #54149
  • [x] <ObserveTyping> #53875
  • [x] <Popover.Slot /> #53889
  • [x] <ShortcutProvider /> #54080
  • [x] <SlotProvider /> #53940
  • [x] useClipboardHandler and CopyHandler #54207
  • [x] useBlockSelectionClearer and BlockSelectionClearer #54209
  • [x] useTypewriter and Typewriter (check if we should bundle or not)
• [ ] Rendering the block editor requires including the stylesheets of a couple packages at least (@wordpress/components and @wordpress/block-editor) even if we're not directly depending on one of these packages (components). Is there a way we can auto-inject these styles while style producing valid npm packages. If not, clarify the documentation.
• [ ] The block-library package provides a set of core blocks to be used by third-party block editors. The problem is that the package doesn't make a distinction between WordPress specific blocks and generic blocks. And even for generic blocks, there might be WordPress specific features. Find a way to clarify which blocks are generic and independent of WordPress and which are not, provide a way for platforms to provide platform specific behaviors on top of core blocks.
• [ ] A lot of recent features of the block editor (theme.json and block supports) rely on server side rendering to generate the frontend styles, even for static markup and static blocks. This means that our serialization function (serialize) doesn't produce an HTML that is ready to be rendered. Provide a way to generate the final HTML including styles (some kind of render function) that can be used by third-party editors. This code already exists and is being used to generate these styles in the editor, but we should make sure to document properly for third-party usage. #54047
• [x] We have a guide to build a custom block editor in the handbook, the guide is tailored towards custom block editors within WordPress which is a small set of third-part block editors, should we consider a dedicated website for "Gutenberg as a Framework" which could include a way to build block editors in any JS application, build universal block types (block types that are WP independent)... #54375
  • [x] Start a new documentation website with a "Getting started" guide for usage of Gutenberg as a framework
  • [x] Fill all the documentation pages
  • [x] Improve the design of the home page
  • [x] Deploy the website
• [x] Undo/Redo is bundled into core-data making harder for third-party block editors to use, they have to implement their own implementation. Extract that into a reusable package and offer docs on how to hook it with the block editor. #54377
• [x] Simplify React usage and reconsider the @wordpress/element abstraction. #54074
• [ ] Color palettes not working properly in the editor #55036
• [ ] Provide sane defaults for all block editors https://github.com/WordPress/gutenberg/pull/56483/files#r1403583859

cc @WordPress/gutenberg-core

[spencerfinnell]

https://github.com/Automattic/isolated-block-editor/ is likely a good indicator of the additional scaffolding currently needed to use Gutenberg in a more isolated environment.

https://github.com/WordPress/gutenberg/discussions/46693 has discussions about the potential advantages of reducing the footprint needed to utilize Gutenberg/WordPress packages.

[mtias]

Thanks for starting this!

[andrewserong]

+1 thanks for writing this up!

Block supports and server side rendering: A lot of recent features of the block editor (theme.json and block supports) rely on server side rendering to generate the frontend styles, even for static markup and static blocks.

It could be good to break this out into a separate issue, but just adding a couple of thoughts and previous discussions for this one:

• https://github.com/WordPress/gutenberg/discussions/37495 — discussion of trade-offs of storing styles in post content versus dynamically injecting styles at render time
• https://github.com/WordPress/gutenberg/pull/39417#issuecomment-1073568618 — discussion about ignoring changes to style when validating blocks, and a comment about moving toward server-rendering of inline styles

Can we solve this by introducing a "styles engine" that has two implementations: one on the server, one on the client. The client implementation is opt-in (used in the editor and also can be used by third-party block editors to render their blocks as needed), the server implementation is preferred for the frontend in WordPress. (From https://github.com/WordPress/gutenberg/discussions/37495)

From those discussions, it sounded like dynamic injection was preferable for a few reasons:

• Avoid problems of deprecations when output changes
• Allow dynamically switching value output on and off (i.e. via themes disabling particular output, etc)
• Allow values and output logic to be filterable by themes or plugins (or 3rd party editors)

Those discussions also raised the downside that 3rd party editors would need their own approach for outputting dynamic styles. So, it's good to raise that here. I like the idea of a canonical approach for 3rd party editors to "switch on" rendering output of block supports in some way via a render function.

Overall, is the goal to still move toward a state where styles are all generated dynamically, or is it more of a case-by-case scenario depending on the kind of block supports we're designing/working with?

Provide a way to generate the final HTML including styles (some kind of render function) that can be used by third-party editors.

In terms of how we define when to dynamically inject styles, would a potential solution be to have a boolean flag somewhere, where we flag whether or not to output at render time? Some block supports (e.g. layout and elements) in JS are hooked in to output only within BlockEdit, so I imagine there'd need to be an API of some kind for these block supports, so that they can be supported in both contexts (edit and "JS render" 🤔)

[youknowriad]

@andrewserong Please go ahead with the issue, we can link to it from here. I agree that this deserve its own issue. At the moment, I didn't give too much thoughts and I'm not too opinionated. All what I can say is that we should provide third-party editors with a way to retrieve the final HTML for these static blocks (paragraph, headings....) including all the block supports that are not really dynamic (don't depend on any server-side stored value). I don't think the decision to store the styles inline or not need to change, it's fine we're opinionated there per block support but third-party editors (JS only) should be able to render the posts properly without writing PHP and just calling a function we provide.

[ellatrix]

• The block-library package provides a set of core blocks to be used by third-party block editors. The problem is that the package doesn't make a distinction between WordPress specific blocks and generic blocks. And even for generic blocks, there might be WordPress specific features. Find a way to clarify which blocks are generic and independent of WordPress and which are not, provide a way for platforms to provide platform specific behaviors on top of core blocks.

I wonder if each block should be its own package. That's what I was considering for footnotes at first. CKEditor for example does this: https://ckeditor.com/docs/ckeditor5/latest/api/paragraph.html

[andrewserong]

Please go ahead with the issue, we can link to it from here.

Thanks for the extra context @youknowriad! I've opened up a separate issue for this in #54047 and linked it in the issue description. Feel free to edit as needed 🙂

[gziolo]

I wonder if each block should be its own package. That's what I was considering for footnotes at first. CKEditor for example does this: https://ckeditor.com/docs/ckeditor5/latest/api/paragraph.html

It’s already possible to import individual blocks which isn’t that far from as if they were in separate packages and it’s documented in:

https://github.com/WordPress/gutenberg/tree/trunk/packages/block-library#registering-individual-blocks

[youknowriad]

@gziolo oh that's really cool, I missed that I think. it would be cool to have a dedicated entry point in block-library that is for universal blocks only (blocks that don't require server side code to work fully).

[youknowriad]

@ellatrix for the typewriter mode, I decided not to bundle it for now. I hesitated about whether I should add a flag to enable it in BlockCanvas. It's still an option at some point if there's demand for it but for now, I left it as an optional external hook you may decide to apply. There's also an extra padding (40vh) that is added when the typewriter mode is enabled that could also be added automatically if the flag is set to true.

[tellthemachines]

Rendering the block editor requires including the stylesheets of a couple packages at least (@wordpress/components and @wordpress/block-editor) even if we're not directly depending on one of these packages (components).

Not sure I understand, components is a dependency of block-editor, right?

The block-library package provides a set of core blocks to be used by third-party block editors. The problem is that the package doesn't make a distinction between WordPress specific blocks and generic blocks.

Even if we distinguish between WP and generic blocks, block-library has core-data as a dependency so anyone installing it will get that bundled in. Perhaps we could have two packages instead? like generic-block-library and wp-block-library?

[youknowriad]

Not sure I understand, components is a dependency of block-editor, right?

Yes, but when you do import something from '@wordpress/block-editor' the CSS produced by @wordpress/block-editor or @wordpress/components is not imported automatically. That's one of the reasons most UI libraries use CSS in JS instead of shipping separate CSS files.

Our packages do ship CSS files but the consumer has to include them manually https://github.com/WordPress/gutenberg/blob/1cfad2ca1744e8aea15b0d517fbf8e73a509b713/platform-docs/docs/intro.md?plain=1#L83-L90

Even if we distinguish between WP and generic blocks, block-library has core-data as a dependency so anyone installing it will get that bundled in. Perhaps we could have two packages instead? like generic-block-library and wp-block-library?

Yes that's one option, that's the work we need to do: rule out all WP specific dependencies and potentially have a way to "augment" the blocks that are "mixed". In the sense that some blocks are in general "universal" but when they are used within WordPress, they need to check the post or something, so we need to figure out if we can have a way to filter/extend/augment these blocks in the wordpress specific package.

[youknowriad]

Status update:

• I'm really satisfied with the progress we've been making here. We removed a lot of boilerplate from the rendering of the block editor.
• We also now offer a hook to implement undo/redo quickly.
• We have multiple playgrounds now in Storybook to show different examples of block editors.
• We've bootstrapped a documentation website that is specially tailored towards Gutenberg usage outside of WordPress.

There's a lot of remaining work though (and some big undertakings). These are the things I'd love if we could tackle next:

• Explore the "render" functions to generate the full HTML + styles.
• Clarify what blocks are WP agnostic and what blocks are not and how to import the blocks and augment them with CMS specific behavior.
• Continue on the documentation website, fill the remaining pages until it's ready for deployment.

Any help with these things would be highly appreciated.

[xstaticwebdev]

@spencerfinnell asked me to comment on this, so here's my two cents. First of all I'm not a WordPress developer so I'm not intimately familiar with the Gutenberg/WordPress architecture and I don't know the technical limitations therein. As a consumer of "WordPress as a service" I really just need 3 things:

1.) The HTML blob that is generated by the block editor for a given post. This is already available using the rest api. 2.) The css blob that corresponds to the post, ideally minified. As a nextjs user I can then dump the css blob into a styled jsx block and the content will be displayed just as it would be in native WordPress. Other frameworks would have something similar to styled jsx. 3.) The javascript blob associated with a given post. Some blocks could rely on Javascript so the ability to ship that to the front end to run seems important. Yes, this is a little messy, but I'm not sure what other alternatives would be.

So an api route that returns something like this would be ideal:

post: { html: {"

.....

"}, css: {"body:{......}"}, js: {"document.QuerySelector......"} }

I'm not at all sure that this is possible, but it would make building headless sites with WordPress super easy. I'd be willing to test any implementation if/when available.

[youknowriad]

@xstaticwebdev Thanks for the feedback. To clarify a bit, the issue here is not necessarily about Headless WordPress. It's a bit of tangent. It's about using Gutenberg as a framework to integrate to any application or CMS. I'm certain that a lot of work here can help a bit with Headless WordPress but I do believe your use-case deserves a dedicated issue.

It's true that it's not clear at the moment how to retrieve the rendered HTML + CSS + JS using a REST API in WordPress. I'm not sure it will ever be, because a lot of hooks depend on the URL and context that is not always possible to pass to the REST API but at least it's worth exploring.

[kohheepeace]

Hello, In relation to this issue, I thought it would be helpful if a list of required API endpoints and expected return values were documented for core blocks.

*I am currently trying to get gutenberg to work in a ruby on rails environment.

For example, core/embed block needs /oembed/1.0/proxy endpoint which returns oEmbed json response.

I struggled with other issues related MediaUpload which needs to return a value in the appropriate json format after successfully saving an image for core/image block.

And I'm currently tackling to implement reusable blocks, I need to define GET /wp/v2/blocks/<id> and return the record in the appropriate json format.

Anyway, thank you very much for working on making gutenberg usable as a framework. There are countless wysiwyg editors, but I don't think there is any other wysiwyg editor that is so practical as a "site builder" than gutenberg.

[youknowriad]

@kohheepeace oh that's a big undertaking, it's exciting to see where you can take this. While the current issue is about the reusability of the "block-editor" package, I do think we need to tackle the reusability of the "editor" package at a more global level at some point. Which means relying on the "core-data" package as an API abstraction. I'd love if we can at some point offer an official API to build an "adapter" for the "core-data" to connect it within API for persistence (another REST API for instance)

[kohheepeace]

I'd love if we can at some point offer an official API to build an "adapter" for the "core-data" to connect it within API for persistence (another REST API for instance)

=> I can't picture it clearly now, but I'm looking forward to it 👍.

Additionally, along with this, there is another a bit troublesome task: the generation of the global-styles.css and settings.json from the theme.json.

Currently I create theme.json (+ patterns in patterns/ etc...) , then run wp-env start to start wordpress environment and run wp.data.select('core/editor').getEditorSettings() in Chrome Devtools's console to get settings.json and search global-styles css from Element tab.

I hope this process become easier (maybe JS equivalent of WP_Theme_JSON_Resolver or cli ?) 👍

*sorry for my greed to request out-of scope.

[youknowriad]

Status update: We now have a temporary location for the Gutenberg as framework website where we can iterate on.

As discussed here https://github.com/WordPress/wporg-main-2022/issues/379 we might want to move this to wordpress.org/gutenberg ultimately.

[mrfoxtalbot]

Noting that @growthwp's comment is somehow related to this other discussion on how to handle iframes https://github.com/WordPress/gutenberg/issues/59926

[youknowriad]

We have a question in regards to the (lack of?) Iframe component, since you can now use shouldIframe - moving forward, does the team wish to abandon the "iframe container"?

There's no intention of abandoning the iframe container. It's kind of the opposite shouldIframe default to true and it's a private API because we want most block editors to always use the iframe.

[youknowriad]

No problem at all, I am happy to answer questions and clarify what needs to be clarified :)

[vonloxx]

Finally I'm doing the refactoring for the whole Drupal Gutenberg using this framework. The first step is to set it up and check if pretty much all functionality we had from the edit-post component is there (only in terms of the editing experience).

From my initial implementation (following the documentation), here's some issues I'm having:

• Align wide/full doesn't work, doesn't show the options when setting the alignment for a block. I'm using the same settings that works with the current implementation (with edit-post)
• Handler to resize images is not available
• Any way that the iframe auto resizes (height) according to the content?
• Some styles (link) aren't injected in the iframe. Only ones with these ids are injected:
  • wp-block-editor-content-css
  • wp-edit-blocks-css
  • wp-block-library-css
• A request is made to /wp/v2/types?context=view&_locale=user. Is it necessary?

If anyone has any tips to help solve those issues I would appreciate it 😉 Btw, I'm using the branch wp/6.6.

cc/ @youknowriad

[youknowriad]

@vonloxx Thanks for the comment, Do you think it would be possible to have a codesanbox or something like that with an isolated block editor to be able to reproduce and isolate some of these more easily.

[youknowriad]

A request is made to /wp/v2/types?context=view&_locale=user. Is it necessary?

This seems to be related to the footnotes. I left a comment on #51201 with a potential fix.

[youknowriad]

Any way that the iframe auto resizes (height) according to the content?

Did you consider a 100% height instead? to avoid double scrollbars...

[vonloxx]

Any way that the iframe auto resizes (height) according to the content?

Did you consider a 100% height instead? to avoid double scrollbars...

I ended up resolving using a component with a resize observer (and mutation observer). It simply observes the content height of the iframe's body and set the iframe's height accordingly. It works great ;) Btw, this functionality is for when the editor is used as an isolated editor (like CKEditor/TinyMCE). For the full UX experience like edit-post, yep, we'll use a 100% height.

[stokesman]

I ended up resolving using a component with a resize observer (and mutation observer). It simply observes the content height of the iframe's body and set the iframe's height accordingly. It works great ;)

It’s a bit of a digression but I'm curious about this. Is the mutation observer used to limit the responses to the resize observer? I ask because trying to size an iframe by its content’s height tends to create infinite loops when heights of any contained elements depend on vh units.

[vonloxx]

@stokesman Actually adding two mutation observers, one for the closest container of the iframe (to wait for the iframe) and when the iframe is found, disconnect the observer and another observer is added to the iframe to look for the body when available. After the body is found, I add the resize observer to the iframe's body and disconnect the last mutation observer. I didn't try with elements with vh units tho 🤔