Override robots settings from within a component

[rudolfbyker]

Clear and concise description of the problem

Sometimes we would like to modify the robots settings for a specific page or a specific URL, based on some logic that is only available while rendering the app. Use cases:

• If my ErrorAlert component is used on the page, I want to exclude that page from indexing until I fix the error. I can't do this using routeRules.
• If my polyglot user is viewing content in language A while the app itself is set to language B, that page should not be indexed. The URL might look like /en/node/3?lang=fr for French content being displayed on an English app. According to Google's recommendations, such a page should not be indexed (/en/node/3?lang=en and /fr/node/3?lang=fr should still be indexed). I can't do this using routeRules.

Suggested solution

Some kind of composable, e.g. useRobotsConfig(…) which will override both the meta tag and the x-robots-tag header for the current request. This should work in any component, not just the top-level page component, since the ErrorAlert might be shown a bit deeper in the component tree. If useRobotsConfig is never called, then use the routeRules as usual. If useRobotsConfig is called more than once, I'm not sure what should happen...

Alternative

I'm currently using this, but it means that my x-robots-tag header is in conflict with my <meta name="robots"> tag:

useHead({
  meta: [
    {
      // This only affects the robots meta tag. It does not affect the `x-robots-tag` header.
      name: "robots",
      content: "noindex, nofollow",
    },
  ],
});

Additional context

No response

[harlan-zw]

Thank you for the detailed write-up, this makes perfect sense. I'll try and implement when I have a chance.