[feature] Support setting Mastodon-compatible `indexable` account flag

[VyrCossont]

Is your feature request related to a problem ?

As a GtS user with an account being used for project announcements, news articles, or other public broadcast communication, I want to opt my posts into showing up in Mastodon full-text search.

Mastodon has a specific flag for this, having opted to use toot:discoverable only for profile search, and added support for the toot:indexable flag on remote AP actors in mastodon/mastodon#26485.

Describe the solution you'd like.

We would add Mastodon-compatible support for this flag:

• DB: add an indexable column to gtsmodel.Account, defaulting to false
• AP: add toot:indexable property to AccountToAS and ASRepresentationToAccount, alongside toot:discoverable
• client API:
  • add noindex flag to apimodel.Account (yes, Mastodon uses the negation of indexable for one part of its client API; no, I don't know why)
  • add indexable to the parameters for PATCH /api/v1/accounts/update_credentials
• web UI: add a new "☑️ Mark posts as indexable by search engines" checkbox to /settings/user/profile, under the existing "☑️ Mark account as discoverable by search engines and directories"
• docs:
  • document the new setting, making it clear that it controls post search instead of account search and that only Mastodon is likely to respect it
  • update the existing docs for Mark Account as Discoverable by Search Engines and Directories to remove the existing info box about "searchability", which would now be controlled by the new setting

Describe alternatives you've considered.

The status quo results in GtS posts never showing up in Mastodon search because Mastodon treats a missing indexable as not opting in.

We could make indexable the same setting as discoverable, but GtS has generally fine-grained privacy controls, and I see no real reason to change that here.

Additional context.

This change wouldn't change the behavior of our own search in any way. We can revisit that if we make search more powerful in the future, but right now a user can only search their own posts and replies to them, and further restricting that is pointless.

The change would affects only the behavior of remote Mastodon instances. As far as other implementations, Akkoma and Iceshrimp.NET don't care about indexable, while Pixelfed is capable of setting the flag on its own accounts but doesn't seem to use it otherwise, so this would likely only affect Mastodon instances. It wouldn't affect Google or other web crawlers since we already use discoverable to control robots meta tags on account pages.

[amiskelly]

add a new "☑️ Mark posts as indexable by search engines" checkbox to /settings/user/profile, under the existing "☑️ Mark account as discoverable by search engines and directories"

Just to clarify, would this allow tuning of the <meta name="robots" /> element on status pages?

It's currently <meta name="robots" content="noindex, nofollow">, and I can't see a way to open that up for instances that wish to be fully discoverable and indexable (and even have links followed) by search engines, machine learning, etc. Apologies if I've missed that in the documentation, though.

[tsmethurst]

@VyrCossont That sounds reasonable :)

@amiskelly You're talking about a different issue there, we'll have to investigate that separately (we probably just forgot something in the existing implementation of "discoverable").

[amiskelly]

Thanks @tsmethurst. I created #3064. I hope that's helpful.