Closed WesleyAC closed 1 year ago
Hmmm, 406 means that content negotiation is failing and GtS can't serve anything in response. What are you setting for your content-type
on GET
requests to AP resources?
edit: sorry, I mean what content type are you setting in your Accept
header
Looks like we have Accept: application/json; charset=utf-8
, which I guess should probably be ld+json
instead?
Aha! Yeah, I think it should be something like what's described here: https://www.w3.org/TR/activitypub/#retrieving-objects
So application/ld+json; profile="https://www.w3.org/ns/activitystreams"
EDIT just double checked and we also serve AP when Accept is application/activity+json
Great, thanks for the help! Just put up a PR in Bookwyrm to fix this, will close this issue here :)
You're welcome! Lemme know if there's anything else, would be happy to help :)
Reopening this, since after fixing the Accept header, we're seeing a 401 when trying to make requests to GtS. How can we get more info on what GtS doesn't like about the request? See https://github.com/bookwyrm-social/bookwyrm/pull/2800 for details.
Any pointers on figuring this out?
cc @hughrun
I've done some more testing of this and it appears to be something to do with Bookwyrm signatures not being accepted by GtS. If I send a message from GtS to Bookwyrm, Bookwyrm tries to access the GtS user's account info (to check the GtS user's signature in order to verify the signed digest) but GtS is returning a 401
to this request. This in turn means Bookwyrm can't authenticate the original signed request so it returns its own 401
. Unfortunately the GtS logs aren't giving me much to go on, and I'm not familiar with either the GtS codebase nor with Go, so any pointers we can get on this would be great.
I've been poking at it myself this morning and indeed you seem to be right about GtS not accepting Bookwyrm's signature. I looked in here and this all looks OK to me: https://github.com/bookwyrm-social/bookwyrm/blob/main/bookwyrm/signatures.py#L25. So I'm not sure yet what the issue is. I'll try to gather some trace logs and see what's happening.
Did you already get Bookwyrm working with secure mode mastodon btw? I tried following some bookwyrm accounts from ondergrond.org (which uses secure mode), and that also didn't seem to work, as far as I can tell. So there could be a common issue there, unless I'm mistaken (which is v. possible!).
Did you already get Bookwyrm working with secure mode mastodon btw?
The latest Bookwyrm definitely works with secure Mastodon, but this is a fairly recent update so some servers may not be up to date.
OK, thanks for the confirmation! We (GtS) are going on holiday soonish until the end of this month, so this is likely something that will have to wait til start of May. Keen to get GtS and Bookwyrm playing nicely together.
I got a bit further on the Bookwyrm side of this, and I think I found the reason GTS isn't validating the signatures from Bookwyrm.
With that patch applied, I get the following on my test instance when trying to follow:
gotosocial | timestamp="09/04/2023 18:27:46.014" func=federation.(*federator).AuthenticateFederatedRequest level=DEBUG requestID=p0r4csw704000wywd66g msg="error parsing public key http://bw.builder.hyperborea.org/user/bookwyrm.instance.actor#main-key: cannot find publicKey with id: http://bw.builder.hyperborea.org/user/bookwyrm.instance.actor#main-key"
And when I hit that endpoint on the bookwyrm instance, I get the following JSON:
{"id": "http://bw.builder.hyperborea.org/user/bookwyrm.instance.actor", "type": "Person", "preferredUsername": "bookwyrm.instance.actor", "inbox": "http://bw.builder.hyperborea.org/user/bookwyrm.instance.actor/inbox", "publicKey": {"id": "http://bw.builder.hyperborea.org/user/bookwyrm.instance.actor/#main-key", "owner": "http://bw.builder.hyperborea.org/user/bookwyrm.instance.actor", "publicKeyPem": "-----BEGIN PUBLIC KEY-----\nMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEA13GWgPpnbQUH+jT5dy3G\nQxUGuMgcL7GzkYZRtT6RIX++H/7fMhxRX2nxqqEWC/bc/tioZbrABtjRD4otVqLe\nghEkHCakwOVzYnyBs6Be7Nw7DYGsuBVmirgcVG8SK0bfoVowv069q+vei3VZlqeZ\nJobhFs+sLf0HAFWJbtLk1gvH3DMz2GJKDBSoFAK5hYsVLq4NpybynhvSi09ZlQEc\nRPCCe0YaDvecScQbRHIsOVseHVn5M3Zi9AviTwx2qgkEw7nWeDNMURFDd4QWquAZ\nI7nEOLO43fkxu8ohiY+3EddXW39VY61c7KQAx5ITRil8JMa+jLeE0IIGXSmN0kPj\nuQIDAQAB\n-----END PUBLIC KEY-----"}, "followers": "http://bw.builder.hyperborea.org/user/bookwyrm.instance.actor/followers", "following": "http://bw.builder.hyperborea.org/user/bookwyrm.instance.actor/following", "outbox": "http://bw.builder.hyperborea.org/user/bookwyrm.instance.actor/outbox", "endpoints": {"sharedInbox": "http://bw.builder.hyperborea.org/inbox"}, "icon": {}, "bookwyrmUser": true, "manuallyApprovesFollowers": false, "discoverable": false, "hideFollows": false, "@context": ["https://www.w3.org/ns/activitystreams", "https://w3id.org/security/v1", {"manuallyApprovesFollowers": "as:manuallyApprovesFollowers", "schema": "http://schema.org#", "PropertyValue": "schema:PropertyValue", "value": "schema:value"}]}
http://bw.builder.hyperborea.org/user/bookwyrm.instance.actor#main-key
- What GTS is looking fow
http://bw.builder.hyperborea.org/user/bookwyrm.instance.actor/#main-key
- What the JSON identifies the key as.
If GTS is looking for the exact ID in the JSON, that's probably where the communication is breaking down.
I'm not familiar enough with the spec to say which side should be normalizing this, though I'd tend to go with the robustness principle and say that both should.
Ok I have almost fixed this!
tag
value as a single object rather than a list if there is only one user tagged (mentioned). Whilst the spec is slightly ambiguous, given the example they provide is a single tag object wrapped in a list, it seems to be that GtS is non-compliant in this regard. On the other hand, I suppose the Bookwyrm team could just suck it up and build in some logic to account for the tag
value being either a list/array or a single dict/object. In any case, there still seems to be an issue where GtS isn't understanding the tag user value Bookwyrm is sending.Ahhh thank you for the investigation and the fixes! Awesome :heart_eyes:
I think in the case of sending tag
as a single object when there's only one, GtS is in-line with ActivityStreams (see example 157 here -- https://www.w3.org/TR/activitystreams-vocabulary/#microsyntaxes). We recently fixed an issue where we were wrongly serializing orderedItems as an object when there was only one of them (https://github.com/superseriousbusiness/gotosocial/pull/1673#event-8944845917), but that seems to have been an exception because orderedItems is a special case. I could be wrong though, I struggle to follow json-ld :grimacing:
@tsmethurst all good, I think I have dealt with that at our end - it's a bit unclear to me which is the correct behaviour but both make sense.
The last piece of the puzzle (since every time I fix something I find a new, exciting bug) is that there are two problems which I suspect have the same root cause:
As far as I can tell, these messages are received and accepted both by GtS and Bookwyrm, but both systems then just don't notify mentioned users and don't deliver DMs to user inboxes. This is quite weird, and I'd appreciate any tips you might have (noting that you've told us you'll be on leave soon, so if we have to wait, we have to wait 😄 ).
Oh in case you're interested, my fixes so far are at https://github.com/bookwyrm-social/bookwyrm/pull/2812
As far as I can tell, these messages are received and accepted both by GtS and Bookwyrm, but both systems then just don't notify mentioned users and don't deliver DMs to user inboxes. This is quite weird
that is super weird! could you post an example of a bookwyrm post with a mention in it? then (after break) we can add it to our GtS tests and see if we can figure out what's happening...
I figured it out!!!
GtS expects the name
field in a mention tag
to be formatted like @user@example.tld
but Bookwyrm is sending them as user@example.tld
. When I change that, everything works.
I think GtS should accept either format, but we can make changes at the Bookwyrm end too.
There's a small remaining problem but that has nothing to do with GtS. Thanks for helping talk this through.
I think GtS should accept either format
agreed!
Thanks for helping talk this through.
My pleasure! :heart_eyes:
Gonna close this from our side since it's fixed in https://github.com/bookwyrm-social/bookwyrm/pull/2812 <3
Describe the bug with a clear and concise description of what the bug is.
As reported by @kvibber in https://github.com/bookwyrm-social/bookwyrm/issues/2794, searching for a GTS user from a Bookwyrm instance results in a error, stemming from GTS giving a 406 error in response to Bookwyrm's signed request. It's fairly likely the error is on Bookwyrm's side, since signed requests are fairly new for us, but I'm opening this issue here to get some feedback, in case it's obvious what's going wrong.
Our signatures for this request have a
keyId
that looks likehttps://bookwyrm.social/user/bookwyrm.instance.actor#main-key
, and a algorithm ofrsa-sha256
, you can see the code we use to generate them inbookwyrm/signatures.py
.What's your GoToSocial Version?
0.8.0-rc1 git-e46323c
GoToSocial Arch
No response
What happened?
No response
What you expected to happen?
No response
How to reproduce it?
Search for a GTS user from any Bookwyrm instance running Bookwyrm v0.5.5 or later.
Anything else we need to know?
No response