Content analysis unable to parse Blockeditor dynamic blocks

[davecpage]

• [x] I've read and understood the contribution guidelines.
• [x] I've searched for any related issues and avoided creating a duplicate issue.

Please give us a description of what happened.

For our client we have developed a number of custom Gutenberg blocks and now that they are building pages using these blocks we have discovered that the Yoast SEO plugin v does not seem to be able to "see" them as part of its content analysis. We have switched to the Premium version of the plugin and the same issue occurs. The plugin does not seem to have a problem "seeing" the content of standard WP blocks.

We have not tested this using a default WP theme as the custom blocks would no longer be available to render the block content. They are all dynamic blocks rendered in PHP.

Please describe what you expected to happen and why.

We expected for Yoast to be able to see the content of the custom blocks.

How can we reproduce this behavior?

1. Use a dynamic block, such as https://developer.wordpress.org/block-editor/tutorials/block-tutorial/creating-dynamic-blocks/

Technical info

• If relevant, which editor is affected (or editors):
• [ ] Classic Editor
• [x] Gutenberg
• [ ] Classic Editor plugin

• Which browser is affected (or browsers):
• [x] Chrome
• [x] Firefox
• [ ] Safari
• [ ] Other

Used versions

• WordPress version: 5.2.2
• Yoast SEO version: 10.1.3 and Premium: 11.6
• Gutenberg plugin version: 5.7.0
• Classic Editor plugin version:
• Relevant plugins in case of a bug:
• Tested with theme: Custom theme for client

[Djennez]

Hi @davecpage and thank you for your question.

I just created a custom block based on this guide: https://getflywheel.com/layout/wordpress-gutenberg-blocks-custom/ . Adding words to the block successfully increased the word count in our analysis.

If the block renders the content to be analyzed in the backend it should not have any issues. If it is a block that relies on dynamic content it would be very hard to analyze. Based on your report I'm not really sure what exactly you're looking to accomplish. Can you elaborate this a bit more?

[davecpage]

Hi @Djennez

We used the guide https://developer.wordpress.org/block-editor/tutorials/block-tutorial/creating-dynamic-blocks/ for dynamic blocks which has save functions returning null as they are not used on the frontend. This is what causes Yoast to not "see" the content, it uses the save function rather than what is outputted and seen visually.

We worked around this by having a save function returning a simplified version of the server-side render (i.e. converting everything to simple headings and paragraphs). It's not something that editors would see, and even though it's stored in post_content it's replaced on the frontend by the actual server side render function, but it means there is something for Yoast to analyse.

It just seems a roundabout way of doing it. And it means that what is stored in post_content does not necessarily reflect what the final result will be, such as a dynamic collection of related resources. It is possible for Yoast to be able to analyse what is seen visually to the editor? Therefore not requiring a save for dynamic blocks?

[vicolaspetru]

I have the same issue with Yoast Analysis, because of dynamic blocks which has save function returning null. Is there a way to fix it?

[MatthewEppelsheimer]

+1 to addressing this issue.

This is or will soon become a widespread issue, since it affects dynamic blocks that are created with the best practices documented on developer.wordpress.org (@davecpage linked to this above). Outputting content on save may be a short-term workaround, but doesn't seem like a scalable long-term solution.

I imagine a solution will involve adding a hook to Yoast that developers can use to make Yoast aware of custom dynamic blocks. Developers would need a way to pass content from block render callbacks in PHP to Yoast. There would also need to be a JavaScript solution for Yoast's live analysis tools to work with Gutenberg component save methods.

[pascalvangaal]

+1, Would like such a feature indeed!

[DB-Alex]

I think this is related to this also?:

https://github.com/Yoast/yoast-acf-analysis/issues/241

[monbauza]

Please inform the customer of conversation # 591569 when this conversation has been closed.

[CharlieTyler]

+1, I would also like this feature added! Surely use of dynamic blocks will only increase.

[mmikhan]

Please inform the customer of conversation # 611830 when this conversation has been closed.

[ghost]

Yoast is not the only SEO analyzer thing having a problem with Dynamic Blocks. In fact I was told by The SEO Framework developer that dynamic blocks were a step back ward in web development :) and dynamic blocks shouldn't be used. (-1)

The real issue is that they are reading the editor content with a JS call something to the effect of:

wp.data.select('core/editor').getEditedPostAttribute('content')

While Yoast and other plugins think this is an effective way of analyzing page content it is not.

Even core blocks like latest posts don't render HTML in the editor...

A much better solution would be to generate a preview or just read the actual HTML output of the wordpress page with an ajax call or similar. Dynamic blocks are a real thing, they're approved and encouraged by Wordpress.org. Don't be fooled by amateurs claiming they are not and you have to refactor and develop your site so it can be read by Yoast or TSF, Yoast and TSF need to fix THEIR problem.

[sybrew]

I believe dynamic blocks are a step backward because I foresee a future that brings us back to the days of shortcodes.

Nevertheless, they do have their legitimate uses (ads, latest posts, author bio, displaying any content from a source outside of the page, etc.), but they aren't useful to be parsed by SEO plugins.

I only hope these dynamic blocks won't be used for creating static content like many page builders have done with shortcodes in the past.

From Gutenberg's documentation:

There are two primary uses for dynamic blocks:

1. Blocks where content should change even if a post has not been updated. One example from WordPress itself is the Latest Posts block. This block will update everywhere it is used when a new post is published.

The second use is the same as the first one. But, then aimed at HMTL markup instead of content. If you mean to create static content via a dynamic block, then I believe you're taking the wrong approach.

You can register a dynamic block by setting the save value to null or omitting it completely:

wp.blocks.registerBlockType( name, { save: null } )

Because no saving occurs, the block's content won't be stored in Gutenberg's data handler, and we cannot access it. Regardless, because the content is evergreen, the validation data from SEO plugins can be invalidated at any time.

It'd be best for SEO plugins to ignore this content because users may get a false sense of security.

[jdevalk]

The problem with analyzing dynamic blocks is inherent in their name: they're dynamic. So technically the analysis could change on each page load. That's why they are ignored, because we can't reliably analyze them. Whether we do that by getting the content in the editor or doing calls to the frontend: they're still going to be able to change on every pageload and thus can't lead to reliable and consistent analysis.

If you're using dynamic blocks for non-dynamic content, you're simply doing it wrong in our eyes. Others are welcome to disagree with that, and as seen above, some people do actually disagree with that, but that doesn't mean we have to change our opinion :)

[ghost]

I'm really sorry but as long as there is the filter "the_content" you can't legitimately make this argument. Anything can be changing (for better or worse) the html output before it is put out to the browser. To say we're going to dictate how devs make blocks or to say we only consider what is inside a textarea html control to be the only content of a page/url is bogus. Real SEO analysis has to be on the final rendered output of the page.

There are hundreds of very good SEO analytics tools out there that analyze the final rendered html of website. Most of which don't even need to be installed inside the CMS/Server so if this really is going to be your stance on the subject I am more than happy to uninstall and save the CPU cycles and memory your plugins consume.

[jdevalk]

@brownrl the same is true for adding content with the the_content filter as what I wrote above. You can't do consistent analysis if your content changes all the time. This doesn't change regardless of whether you check from the frontend or the backend.

We differ in our opinion of what "real SEO analysis" is, which is fine. What you don't take into account and we do, is the fact that search engines are smart enough to take a lot of your HTML template and discard it, because it's the same on every page. Lots of SEO tools aren't too good at registering what is truly the content of the page.

I've not even touched on how this relates to Schema on a page, but well, I don't think we're wrong and so far you've not given one compelling argument to change my thinking.

[NicPeer]

Hello,

Just for this discussion: I use Lazy Blocks (great plug-in by the way) as a smart way to structure content in a post on a website that is still under construction (aixpatria.com). I have created a custom block as a template which I can re-use for new posts. I could have done that in HTLM to reach the same ('Yoastable') result of course but now I save a whole lot of time.

Moreover, the content on each post (resulting HTML) I create with the Lazy Block does not change all the time. So SEO analysis what is in the custom blocks would be greatly appreciated (and is needed). So I am not technically proficient enough to understand the argument above about 'real SEO analysis'.

I do not know of other (non-dynamic) solutions that can help me in the same, simple and very effective way. This has however rendered Yoast completely useless for this site and forces me to look at other SEO solutions. I'd prefer to stick with Yoast since for my other sites it works fine.

Nicolaas

[ghost]

I am not the one who's livelihood depends on the sales of the Yoast SEO Plugin, so forgive me if I don't make a compelling argument for anything regarding Yoast. There are 10000s of legitimate reasons why dynamic blocks are used and need to be accepted as normal Wordpress practice.

In the case of lazy blocks the developer is extending and enhancing on the core idea of custom blocks and putting that power in a non programmer/developer hand. However, lazy blocks is not the only one out there to do this sort of thing. In fact Block Lab (https://getblocklab.com/) which is actually being considered to be included in WP Core does something very similar.

Just because the name of the type of block is "Dynamic Block" doesn't mean the content changes all the time. As a simple example a Dynamic Block could simply pull in the information from a 3rd party API and display some information (ex the information of book, movie, recipe. Think the cast from the movie, "Ghostbusters"). That is very cacheable and it is not going to change on each page load let alone day to day.

No one is asking you to do analysis on a live weather forecast... However you can not legitimately say that you are doing SEO analysis on the content of a web page when in reality you are only doing analysis on the "text" inside a "textarea" inside the admin area of the CMS.

To fix this problem here is how I think it can be done:

1) take the content in the editor that you already have 2) send it to an api end point in wordpress 3) process the content the same way as the get_the_content() function WP core does eg. apply_filter('the_content') 4) return the rendered HTML 5) analyze that

5 lines of JS and about 5 lines of PHP, done...

If am completely wrong on how to fix it and it's really not possible, then fine I guess Woorank or SEO Tester Online or ... have to be used instead.

[sybrew]

Just be cause the name of the type of is "Dynamic Block" doesn't mean the content changes all the time.

But that's its sole intention.

1. take the content in the editor and send it to an api end point in Wordpress
2. receive the content
3. process the content the same way as the get_the_content() function WP core does eg. apply_filter('the_content')
4. return the rendered HTML

The Block Editor is WYSIWYG. So, if it can't return the rendered HTML in real-time, why should other plugins resolve that? If each plugin that wants to render dynamic blocks creates a custom solution to this, we'll have a big slow combobulated mess on our hands. Imagine that 40+ plugins (yes, that's normal) make independent calls to your server whenever you blur a block.

Please keep in mind that filters such as the_content should be used for markup (e.g., transforming quotes to curly quotes). If you filter that beyond the block/classic editor's WYSIWYG capability, then your users will have to deal with an unexpected outcome. Yet, it's exactly what the Block Editor wants to resolve. Please refer to Mr. Mullenweg's FAQ on the block editor.

The idea with blocks was to create a new common language across WordPress, a new way to connect users to plugins, and replace a number of older content types — things like shortcodes and widgets — that one had to be well-versed in the idiosyncrasies of WordPress to understand.

He wrote that we need a common language. Not a new language independently created at one's will.

For instance, we could even hook into get_header and show a different page depending on the user agent. But, just because we can, doesn't mean we should. It's impossible to account for this without brute-forcing one's server with requests made with all known user agents. It's a rabbit hole.

Those are things we call "edge cases"--something affecting fewer than 0.1% of our users, and it's something we can't nor shouldn't account for. We'll never get things done that way. If you develop edge-case solutions because you're innovative (more power to you!), then you'll have to deal with the edge-case problems.

If things like Block Lab make it into Core, then it's no longer an edge case. But, the people at Core know precisely how to work around these issues, and they will resolve this if they can, following the same philosophy and principles which we abide by.

In the meantime, I believe you should communicate with the Gutenberg team for registering dynamic-to-static blocks. So that you can support a common language construct, where everything can work in harmony without intervention.

[ghost]

Look, @sybrew you keep referring to your one line of a blog written by one dev of Wordpress which there have been hundreds of people through many years contributing. He is not the end all be all voice of wordpress.org at all...

The block editor is NOT a Wysiwyg. It is a block editor. It edits and manages blocks. Companies like elementor and others have bastardized it and twisted it into a very poor Wysiwyg.

Microsoft front page was a WYSIWYG, Dreamweaver was WYSIWYG, TinyMCE has tried to be a Wysiwyg. Gutenberg is a block editor. The concept of a block editor is actually a real thing, there are numerous JS block editors out there.

The language you need to read and analyze is HTML no one is inventing a new language, the problem is you are assuming the HTML is purely in the "Block Editor" it is not... the HTML is in the final rendered output on the webserver.

Heck I could even write an nginx module or apache module that changes every instance of the word "elephant" with "horse" and as long as you reading the actual rendered output it won't matter to you.

You do understand I am trying to help you, help you read and analyze the actual output that is going to the browser. Making both Yoast and TSF better products?

The text area behind the block editor simply gets stuffed into a text blob in the database. There is no rules that says it must be static HTML 5 iso blah blah... It is a property of the WP_Post object. There are no rules in wp.org that say it must adhere to ...

Using a dynamic block to show the summary of a movie from IMDB is not an edge case. Using a dynamic block to pull in the description of a hotel from a massive API is not an edge case. It's actually how most major websites get made these days.

Let's take a very simple example: Mercedes Benz Automobiles

Everyone of their dealerships has their own website. I know this because... Do you think each and every Mercedes dealership writes and rewrites, copies, pastes, the marketing description of each model of their cars? HELL NO! In fact Mercedes Benz corporate in Germany dictates that they must use a behind the scenes api for the information and descriptions of the models.

THAT is a Dynamic Block my friend. No one is copying and pasting that. Are you going to tell the owner of a Mercedes dealership they can't have SEO analysis on the Series A cars because the "text" is coming from a dynamic block?

[manathan-olivet]

I have the same issue with Yoast SEO Analysis. I use themes with dynamic blocks and the content is not taken in consideration. Can you please resolve this or tell us how to? It's very frustrating.

[naturskyddsforeningen]

Still no ETA or plan for this. Kind of serious problem for us and sites.

[sinclairbenj]

This is a serious issue, we need a solution. Dynamic content has to be readable by Yoast.

[marcoluzi]

I think this is still a problem that should be fixed.