[bug] Failing to fetch data from threads.net

inetol commented 9 months ago

Describe the bug with a clear and concise description of what the bug is.

As you may already know, Meta is carrying out some controversial tests on Threads with some users to make their posts (?) readable through ActivityPub. When trying to search any federated Threads user account via GoToSocial, the instance (GTS) responds with an HTTP 500 error. (detailed issue details appears in the instance logs, see below...)

What's your GoToSocial Version?

v0.13.0 git-f4fcffc

GoToSocial Arch

x86_64 Docker (podman)

What you expected to happen?

The user "should" be accessible and the request would return the user information.

(You can try searching for it on mastodon.online or any Mastodon instances where threads.net is not blacklisted)

How to reproduce it?

Use a web client or mobile app to interact with the GoToSocial instance
Look for some Threads user accessible through ActivityPub (e.g. @mosseri@threads.net)

Request will fail with this error on the instance (GTS) logs:

errors="Error #01: Get: error searching by namestring: accountsByUsernameDomain: error looking up @mosseri@threads.net as account: enrichAccount: error resolving accountable from data for account https://www.threads.net/ap/users/mosseri/: ResolveAccountable: error unmarshalling bytes into json: invalid character '<' looking for beginning of value\n" requestID=ngwz1pcc0XXXXXXXXXXX msg="Internal Server Error: wrote 54B"

Anything else we need to know?

I know this issue involves a third party with proprietary software and the pain in the ass that goes with it, but I hope it will serve as a reference for future users who experience the same problem.

tsmethurst commented 9 months ago

This is a threads bug afaik. They're returning HTML in response to valid ActivityPub json requests, for whatever reason. Nothing we can do about it from our side.

The 500 error on the other hand, we should do something about, because that's a bit weird. But I'd rather have a separate issue for that.

inetol commented 9 months ago

Once the Threads federation is GA and if the issue persists I would like to look at it further, for the moment this is a bit up in the air.

tsmethurst commented 9 months ago

I would be curious to know why something isn't working, so do feel free to gather that info and report it back here for documentation's sake. That said, we're not really interested as a team in putting any effort in to debugging Meta's implementation for them, or in implementing any workarounds for them. If it doesn't work it's up to them to fix it, in other words. They certainly have the resources. I wrote a bit about it from my own account here if you're interested: https://gts.superseriousbusiness.org/@dumpsterqueer/statuses/01HHSFN874A3RSJPKXJCX7RD9E

inetol commented 9 months ago

That's quite an interesting post, thanks!

I've been checking the Threads ActivityPub endpoint and comparing the implementations of other server software instances until I came across this Pleroma issue, is it possible that Threads is filtering GTS requests because it doesn't comply with the AP spec to the letter? (I want to imagine that other implementations that are this strict also give problems with GTS?)

tsmethurst commented 9 months ago

Unlikely imo. We use the instance Actor for barely anything, unlike other implementations. Here's what a GoToSocial instance actor looks like:

{
  "@context": [
    "https://www.w3.org/ns/activitystreams",
    "http://joinmastodon.org/ns",
    "https://w3id.org/security/v1"
  ],
  "discoverable": false,
  "featured": "https://gts.superseriousbusiness.org/users/gts.superseriousbusiness.org/collections/featured",
  "followers": "https://gts.superseriousbusiness.org/users/gts.superseriousbusiness.org/followers",
  "following": "https://gts.superseriousbusiness.org/users/gts.superseriousbusiness.org/following",
  "id": "https://gts.superseriousbusiness.org/users/gts.superseriousbusiness.org",
  "inbox": "https://gts.superseriousbusiness.org/users/gts.superseriousbusiness.org/inbox",
  "manuallyApprovesFollowers": true,
  "name": "gts.superseriousbusiness.org",
  "outbox": "https://gts.superseriousbusiness.org/users/gts.superseriousbusiness.org/outbox",
  "preferredUsername": "gts.superseriousbusiness.org",
  "publicKey": {
    "id": "https://gts.superseriousbusiness.org/users/gts.superseriousbusiness.org/main-key",
    "owner": "https://gts.superseriousbusiness.org/users/gts.superseriousbusiness.org",
    "publicKeyPem": "-----BEGIN PUBLIC KEY-----\nMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAwDgpQpA+CHbJTvLKfgzv\njE9K8C+9O/r7Yj1FxXsXvFL3HxCBtuoIMTCpcgR0A4UK7HIlcFesj5BsvVzFGBsP\nER38D7tkalrqNptTILVw5dBQrXb4ecKbTaamwTZfqR3Q645+n3x0OfQ6EIhTvAK1\nfvjc8+lifmaCjylDQArp3uHOqkdufoKPl6RM2F8W9BY6S+JOGfIqitOjpn38qqBa\noOSWCwZ6/HyDzwrBEK5AvY9gnFwp9uMIP4rhnIyalOCoxaxJR/HZwQmqAS6LS+s6\n810m1kggwfnVt9YS/PHx6ARGLDgXFGK+BXrZjgcisedOxpABDXnw62rrT1mK+9+m\nQQIDAQAB\n-----END PUBLIC KEY-----\n"
  },
  "tag": [],
  "type": "Person",
  "url": "https://gts.superseriousbusiness.org/@gts.superseriousbusiness.org"
}

Our outgoing Accept header looks like this btw:

Accept: application/ld+json; profile=\"https://www.w3.org/ns/activitystreams\",application/activity+json

I wouldn't be surprised if Threads was not correctly parsing that Accept header. That would explain why they're serving HTML in response (though they should probably serve Not Acceptable, but that's another story).

But yeah, like I said, we're not going to put time and effort into debugging this for them. We've got better things to focus on.

daenney commented 9 months ago

I'm going to close this as #2550 should fix this problem. It'll get release with 0.14, or can be tried out with a current snapshot build.

tibequadorian commented 9 months ago

So this issue is not really fixed, but the error is just handled differently?

tsmethurst commented 9 months ago

this issue is not really fixed

As I wrote above, this looks like a threads bug, so not something we can (or ought to) fix from our side. The 500 error from GtS on the other hand we can (and did) fix. And also:

we're not really interested as a team in putting any effort in to debugging Meta's implementation for them, or in implementing any workarounds for them. If it doesn't work it's up to them to fix it, in other words

If Facebook had an issue tracker for threads, someone could open an issue there. If threads was an open source project, we could look at the code and immediately know what's up, but... :shrug:

superseriousbusiness / gotosocial