Force robots to use `noindex,nofollow` when `APP_ENV` is not set to `production`

[aaronbushnell]

Is it possible to force Advanced SEO to use <meta name="robots" content="noindex, nofollow"> if the APP_ENV variable is not production? This would make it really quick for us to disable indexing on staging/development sites.

[aerni]

Not as it stands. But I like the idea.

I'm just a little torn about adding yet another layer of global crawling controls, as we've already got the ability to change the crawling settings globally on a per-site basis. Which makes me think if these controls would even make sense anymore.

What are your thoughts?

[aaronbushnell]

Yeah, I'm absolutely with you—adding more controls can complicate things for a lot of folks 😕.

What if both of these toggles were controllable via the config? And, if they're being modified by the config, they are disabled in the front-end?

I would be completely okay with writing my own check there for something like this if it makes your life easier!

[aerni]

I just opened a PR addressing this issue. What do you think?

I still feel like it adds a disconnect between the config and the visible options in the CP. As a user, you expect the site to be indexed when toggles are off. But the robots tag might be present on the frontend depending on the environment.

One option might be to disable the toggles if they don't have an effect on the frontend. But that might not be desirable either. Maybe a note in the description is the best idea.

[aaronbushnell]

Thanks @aerni! Would something like this be helpful on the front-end? I wonder if—when these are set—if they should completely disable the ability to manipulate them on the front-end?

[aerni]

I've had the same thoughts. I like the visual feedback it provides for the user. However, there are a couple of side effects that are not so ideal.

1. Even if the toggle is read-only, the value will still be saved as true to the file. So whenever you publish the site on production, the site will still be rendering the robots with noindex, nofollow.
2. You might want to "prepare" content in development or staging before they go live in production. Disabling the toggles doesn't give the user the choice to proactively noindex, nofollow the site in general. This might be desirable for bigger sites that take a while to get all the content ready, while working on the content in production.

However, the description text is a feasible solution that will inform the user. I think that should be enough.

[aerni]

What do you think about this? The environment string, e.g. production, is processed from the config on runtime. So the text is updated according to the configured environments.

[aaronbushnell]

I think that looks great, @aerni!

[aerni]

I decided to scratch the explanation. It's a feature for devs and won't really affect end-users.