Instance won't verify my domain link -- Issue now on Mastodon/Mastodon - leaving open

[trickjarrett]

I recently migrated my blog from a subdomain to the root of my domain, however in doing so I needed to reverify my blog link. In updating the URL it is not showing as a verified link.

User @trick@hachyderm.io

[lexfri]

+1'ing this issue. I can't seem to get hachyderm.io to verify the links I've set up, and it looks to me like everything's configured properly.

[quintessence]

👋 Mod here

I wanted to check that you used the information here to verify:

https://github.com/hachyderm/community/blob/main/verification.md

If so, can you share what link / HTML you used?

[lexfri]

I'm at https://hachyderm.io/@lexfri

I posted links to both https://lexfriedman.com and https://daily.lexfriedman.com/

Each has an <a rel="me"> link pointing to my hachyderm.io URL.

[trickjarrett]

I'm at https://hachyderm.io/@trick

The code I'm using, as pulled from my blog:

<a rel="me" href="https://hachyderm.io/@trick">Mastodon</a>

[quintessence]

Something we saw with other users who had trouble verifying for one reason or another was to use the <link> attribute in the header rather than in the body of the web page. The full HTML for that one would look like:

<link rel="me" href="https://hachyderm.io/@username">

Can you try the above and let me know if that works?

[lexfri]

I made this change. Neither of the URLs I updated (lexfriedman.com, daily.lexfriedman.com) worked, but suddenly — and for the first time! — a third URL, reboundcast.com — DID work. The other two still won't verify.

[trickjarrett]

No luck, I added the link line and then resubmitted. I also removed the blog, saved the changes, then went back and added it again, but still no verification.

[joaoasrosa]

I'm observing the same behaviour in my podcast account. I have other accounts on a different server, and I tested to verify the same URL, and it worked.

I intend to join this thread to flag the behaviour and be an extra pair of hands to text potential solutions.

Thanks for all the excellent work!

[trickjarrett]

Just sharing a follow up that I have just tried to verify again now that we are migrated, and still finding it won't verify my site.

[quintessence]

Ah I was going to follow up and ask! We'll try to see if we can figure it out while our bandwidth is freer :)

[trickjarrett]

Interestingly it turns out I was linking to http and it wouldn't verify. But, as soon as I went https, it worked right away.

[joaoasrosa]

In my case, it was already https, and I can see now that it is verified. Maybe the migration process triggered something! Thanks for everything!

[lexfri]

No such luck here. Still can't get https://lexfriedman.com/ or https://daily.lexfriedman.com/ to go verified...

[quintessence]

Glad to hear that it has started working for two of you again!

For @lexfri - I wonder if we try to force it to retry if it'll help. Can you try verifying with:

https://lexfriedman.com/speaking

If that works, then perhaps we can try forcing a re-try on the other two by changing the links and then changing them back.

If not, lemme know and I'll think of something else 🤔

[j3j5]

Something that I noticed with my own and other users is that if you first set up your profile and then change the HTML, the verification doesn't work. You need to have the link ready and then change the URL. Sometimes, removing the url from your profile, saving, and then adding it again does the trick. Actually, I'm not sure if it should be considered a bug, but once you verify the links on your profile, afaics, the check does not run again, that's how I made it so my two profiles are verified against the same websites even though they only point to my hachyderm user. I pointed them to one instance, verified the profile there, then pointed them to hachyderm, and did the same.

[wlonkly]

You need to have the link ready and then change the URL. Sometimes, removing the url from your profile, saving, and then adding it again does the trick

That's the behavior of upstream https://github.com/mastodon/mastodon -- I think if one wanted it to behave differently it'd have to be raised there. But you're right that your own profile checks verification on update. (The view of your profile from federated servers updates on request, I think, with caching.)

[quintessence]

@wlonkly is correct.

Something that we're going to make more clear as we shift the Markdown files to the Community Docs: there are some How Tos that we are documenting for user convenience or our attempt to bridge a gap somewhere, but it'll be ultimately the Mastodon project itself that would need to roll out an individual change.

[lexfri]

I keep deleting and re-adding links to pages with the rel=me and meta tag approaches, to no avail. Most recently tried adding lexfriedman.com/consulting with the account and still no success. Happy to try anything!

[luontola]

I tried adding <a rel="me"> and <link rel="me"> tags for https://hachyderm.io/@luontola, but neither worked when I was linking to my contact information page https://www.luontola.fi/about which contains links to all my social networks. But then when I put to my profile a link to my site's front page https://www.luontola.fi then it started working in a few seconds.

I think this is a bug in the verification logic. Right now it only says "verify yourself as the owner of the links in your profile metadata" which doesn't indicate anything about it having to be a link to the domain root page. If it's meant to work only for the domain front page, there should be a clear warning about it.

[luontola]

I think I found the problem. The verification doesn't work if the URL doesn't end with a file extension.

Verification fails for these URLs: https://www.luontola.fi/about https://www.luontola.fi/about/

Verification works for this URL: https://www.luontola.fi/about.php

My .htaccess is like this:

Options Indexes Includes FollowSymLinks MultiViews
IndexOptions +NameWidth=*

Seems like the verification is incompatible with Apache's MultiViews setting.

[wlonkly]

(Note that Github is inconveniently removing the / from the display, but not the link, of your second example URL.)

I don't think that's it, alas -- I just successfully verified using a long link ending in /. (I'll leave that in place for a while.)

The MultiViews part is interesting, though. What do your Apache logs show with hachyderm in the User-Agent?

After my change above, mine look like

88.99.61.61 - - [17/Dec/2022:19:00:32 +0000] "GET /blog/2022-12-mastodon-verification-test/ HTTP/1.1" 200 4094 "-" "http.rb/5.1.0 (Mastodon/4.0.2; +https://hachyderm.io/)"

plus similar from a bunch of other instances.

[luontola]

Interestingly GET /about returns HTTP 406 whereas GET / returns HTTP 200. 🤔 This requires more investigation.

www.luontola.fi 172.20.0.1 - - [17/Dec/2022:19:57:58 +0000] "GET /about HTTP/1.1" 406 354 "-" "http.rb/5.1.0 (Mastodon/4.0.2; +https://hachyderm.io/)" "-"
www.luontola.fi 172.20.0.1 - - [17/Dec/2022:19:57:58 +0000] "GET / HTTP/1.1" 200 1603 "-" "http.rb/5.1.0 (Mastodon/4.0.2+glitch; +https://infosec.exchange/)" "-"
www.luontola.fi 172.20.0.1 - - [17/Dec/2022:19:57:58 +0000] "GET / HTTP/1.1" 200 1603 "-" "http.rb/5.1.0 (Mastodon/4.0.2; +https://mastodontech.de/)" "-"
www.luontola.fi 172.20.0.1 - - [17/Dec/2022:19:57:58 +0000] "GET / HTTP/1.1" 200 1603 "-" "http.rb/5.1.0 (Mastodon/4.0.2; +https://mastodon.online/)" "-"
www.luontola.fi 172.20.0.1 - - [17/Dec/2022:19:57:58 +0000] "GET /about HTTP/1.1" 406 354 "-" "http.rb/5.1.0 (Mastodon/4.0.2+glitch; +https://infosec.exchange/)" "-"
www.luontola.fi 172.20.0.1 - - [17/Dec/2022:19:57:58 +0000] "GET / HTTP/1.1" 200 1603 "-" "http.rb/5.1.0 (Mastodon/4.0.2; +https://chaos.social/)" "-"
www.luontola.fi 172.20.0.1 - - [17/Dec/2022:19:57:58 +0000] "GET / HTTP/1.1" 200 1603 "-" "http.rb/5.1.0 (Mastodon/4.0.2; +https://masto.ai/)" "-"
...

[luontola]

The previous logs were from my reverse proxy. These are the logs from the actual Apache httpd instance that serves the website. The first request is when I visit the page in my browser, and the others are from Mastadon instances.

luontola.fi:80 172.22.0.3 - - [17/Dec/2022:20:05:04 +0000] "GET /about HTTP/1.1" 200 1647 "https://www.luontola.fi/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/108.0.0.0 Safari/537.36"

[Sat Dec 17 20:05:49.689109 2022] [negotiation:error] [pid 19] [client 172.22.0.3:57252] AH00690: no acceptable variant: /var/www/luontola.fi/html/about
luontola.fi:80 172.22.0.3 - - [17/Dec/2022:20:05:49 +0000] "GET /about HTTP/1.1" 406 611 "-" "http.rb/5.1.0 (Mastodon/4.0.2; +https://hachyderm.io/)"

[Sat Dec 17 20:05:50.268881 2022] [negotiation:error] [pid 18] [client 172.22.0.3:57274] AH00690: no acceptable variant: /var/www/luontola.fi/html/about
luontola.fi:80 172.22.0.3 - - [17/Dec/2022:20:05:50 +0000] "GET /about HTTP/1.1" 406 611 "-" "http.rb/5.1.0 (Mastodon/4.0.2+glitch; +https://infosec.exchange/)"

The error "AH00690: no acceptable variant" is explained at https://stackoverflow.com/questions/16357933/no-acceptable-variant-from-multiviews-in-apache

It seems to be related to the request's Accept header and the workaround is to add some server configuration:

<Files "*.php">
    MultiviewsMatch Any
</Files>

[luontola]

When I visit the page with a web browser, the accept header contains text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8, referer: https://www.luontola.fi/. But when Mastodon does the request, its accept header contains only text/html. 💡

I think I'll go make a bug report and feature request upstream to fix this in all Mastodon servers. Also it would be very useful if the profile page would report any errors and reasons why verification failed. Debugging this would have been much faster if Mastodon told me directly that the server returned HTTP 406. Not everyone has access to the server logs of their website.

[rbairwell]

Just coming from the https://github.com/mastodon/mastodon/issues/22418 issue, I've just checked https://www.luontola.fi/about and https://www.luontola.fi/about/ manually using Brave, https://www.uptrends.com/tools/http-response-header-check and https://dnschecker.org/server-headers-check.php - and received "Content-Type text/html; charset=UTF-8" as the correct header in all instances.

[luontola]

The application/x-httpd-php content type was never shown to the web browser. It was only used internally by Apache. I fixed the server configuration before reporting those issues, so you can't use them to reproduce the issue anymore (the way to reproduce would have been curl -I -H "Accept: text/html" https://www.luontola.fi/about).

Might be a good idea to check the logs of a popular Mastodon server that how often website validations return HTTP 406. That's such a common configuration error that I don't think I'm the only one.

[dmah42]

just wanting to check: is there anything for us (hachyderm) to do on this issue? it sounds like it's a mastodon/personal server config issue rather than hachyderm-specific. is that right?

[luontola]

Yes, it’s Mastodon specific, not Hachyderm. I already created a couple of issues to Mastodon’s issue tracker that would help with similar issues (they are linked to this issue a couple messages ago).