search

bookwyrm-social / bookwyrm

Social reading and reviewing, decentralized with ActivityPub
http://joinbookwyrm.com/
Other
2.23k stars 262 forks source link

Can't follow to or from GoToSocial #2794

Closed kvibber closed 1 year ago

kvibber commented 1 year ago

Describe the bug GoToSocial can retrieve individual posts from Bookwyrm, and can retrieve a Bookwyrm user's profile, but it cannot follow that user.

Bookwyrm cannot retrieve a GoToSocial user's profile at all and displays an error message when I search for one.

To Reproduce On a GoToSocial instance, search for a Bookwyrm user. Then try to follow them.

On a Bookwyrm instance, search for a GoToSocial user. You can search for my test account at @kelsonv@gts.keysmash.xyz

Expected behavior The GTS user should be able to follow the Bookwyrm user and vice versa

Instance bookwyrm.social

Additional context Add any other context about the problem here.

Using GoToSocial 0.8.0-rc1 and bookwyrm.social.

Logs show both of the following are throwing back 401 Unauthorized HTTP responses: GTS' POST request to https://bookwyrm.social/inbox Bookwyrm's GET request to https://gts.keysmash.xyz/users/kelsonv

This looks very similar to a bug I fixed in Takahe where it was rejecting signatures whose algorithm had been obfuscated by labeling it as "hs2019", which GoToSocial does according to https://datatracker.ietf.org/doc/html/draft-cavage-http-signatures-12 https://github.com/jointakahe/takahe/issues/528

In the case of Takahe it was easy to just treat hs2019 as sha-256, since that's the only signature algorithm it accepts. I took a look at bookwyrm/signatures.py and I see Bookwyrm also supports incoming sha-512 signatures, so it'll be a bit more complicated than just adding or algorithm == 'hs2019' in verify_digest.

I can give this a try over the weekend and see if it works and fixes either (or both) issues.

WesleyAC commented 1 year ago

Stack trace for searching for a GTS user from Bookwyrm:

web_1             | OK: /search
web_1             | Traceback (most recent call last):
web_1             |   File "/app/bookwyrm/views/helpers.py", line 78, in handle_remote_webfinger
web_1             |     user = models.User.objects.get(username__iexact=query)
web_1             |   File "/usr/local/lib/python3.9/site-packages/django/db/models/manager.py", line 85, in manager_method
web_1             |     return getattr(self.get_queryset(), name)(*args, **kwargs)
web_1             |   File "/usr/local/lib/python3.9/site-packages/django/db/models/query.py", line 435, in get
web_1             |     raise self.model.DoesNotExist(
web_1             | bookwyrm.models.user.User.DoesNotExist: User matching query does not exist.
web_1             | 
web_1             | During handling of the above exception, another exception occurred:
web_1             | 
web_1             | Traceback (most recent call last):
web_1             |   File "/app/bookwyrm/activitypub/base_activity.py", line 330, in resolve_remote_id
web_1             |     data = get_data(remote_id)
web_1             |   File "/app/bookwyrm/connectors/abstract_connector.py", line 250, in get_data
web_1             |     resp.raise_for_status()
web_1             |   File "/usr/local/lib/python3.9/site-packages/requests/models.py", line 1021, in raise_for_status
web_1             |     raise HTTPError(http_error_msg, response=self)
web_1             | requests.exceptions.HTTPError: 401 Client Error: Unauthorized for url: https://gts.keysmash.xyz/users/kelsonv
web_1             | 
web_1             | During handling of the above exception, another exception occurred:
web_1             | 
web_1             | Traceback (most recent call last):
web_1             |   File "/usr/local/lib/python3.9/site-packages/django/core/handlers/exception.py", line 47, in inner
web_1             |     response = get_response(request)
web_1             |   File "/usr/local/lib/python3.9/site-packages/django/core/handlers/base.py", line 181, in _get_response
web_1             |     response = wrapped_callback(request, *callback_args, **callback_kwargs)
web_1             |   File "/usr/local/lib/python3.9/site-packages/sentry_sdk/integrations/django/views.py", line 67, in sentry_wrapped_callback
web_1             |     return callback(request, *args, **kwargs)
web_1             |   File "/usr/local/lib/python3.9/site-packages/django/views/generic/base.py", line 70, in view
web_1             |     return self.dispatch(request, *args, **kwargs)
web_1             |   File "/usr/local/lib/python3.9/site-packages/django/views/generic/base.py", line 98, in dispatch
web_1             |     return handler(request, *args, **kwargs)
web_1             |   File "/usr/local/lib/python3.9/site-packages/csp/decorators.py", line 19, in _wrapped
web_1             |     r = f(*a, **kw)
web_1             |   File "/app/bookwyrm/views/search.py", line 48, in get
web_1             |     return endpoints[search_type](request)
web_1             |   File "/app/bookwyrm/views/search.py", line 106, in user_search
web_1             |     handle_remote_webfinger(query)
web_1             |   File "/app/bookwyrm/views/helpers.py", line 94, in handle_remote_webfinger
web_1             |     user = activitypub.resolve_remote_id(
web_1             |   File "/app/bookwyrm/activitypub/base_activity.py", line 337, in resolve_remote_id
web_1             |     data = get_activitypub_data(remote_id)
web_1             |   File "/app/bookwyrm/activitypub/base_activity.py", line 395, in get_activitypub_data
web_1             |     resp.raise_for_status()
web_1             |   File "/usr/local/lib/python3.9/site-packages/requests/models.py", line 1021, in raise_for_status
web_1             |     raise HTTPError(http_error_msg, response=self)
web_1             | requests.exceptions.HTTPError: 406 Client Error: Not Acceptable for url: https://gts.keysmash.xyz/users/kelsonv
WesleyAC commented 1 year ago

I'm not super familiar with the code, but it looks as though Bookwyrm is seeing the 401, correctly falling back to a signed request, but then getting a 406 since the signed request isn't what GTS expects. I'm going to open a issue in the GTS repo for that so we can get some feedback from them.

kvibber commented 1 year ago

I've put together a fix for just the hs2019 issue, where it will try to validate an hs2019 signature as sha256 and if that fails, try to validate it as sha512, and then throw the usual error. So Bookwyrm should be able to validate GoToSocial's signatures now.

It still doesn't solve the whole problem, but I think I've figured out why GoToSocial isn't validating Bookwyrm's signatures!

With that patch applied, I get the following on my test instance when trying to follow:

gotosocial | timestamp="09/04/2023 18:27:46.014" func=federation.(*federator).AuthenticateFederatedRequest level=DEBUG requestID=p0r4csw704000wywd66g msg="error parsing public key http://bw.builder.hyperborea.org/user/bookwyrm.instance.actor#main-key: cannot find publicKey with id: http://bw.builder.hyperborea.org/user/bookwyrm.instance.actor#main-key"

And when I hit that endpoint on the bookwyrm instance, I get the following JSON:

{"id": "http://bw.builder.hyperborea.org/user/bookwyrm.instance.actor", "type": "Person", "preferredUsername": "bookwyrm.instance.actor", "inbox": "http://bw.builder.hyperborea.org/user/bookwyrm.instance.actor/inbox", "publicKey": {"id": "http://bw.builder.hyperborea.org/user/bookwyrm.instance.actor/#main-key", "owner": "http://bw.builder.hyperborea.org/user/bookwyrm.instance.actor", "publicKeyPem": "-----BEGIN PUBLIC KEY-----\nMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEA13GWgPpnbQUH+jT5dy3G\nQxUGuMgcL7GzkYZRtT6RIX++H/7fMhxRX2nxqqEWC/bc/tioZbrABtjRD4otVqLe\nghEkHCakwOVzYnyBs6Be7Nw7DYGsuBVmirgcVG8SK0bfoVowv069q+vei3VZlqeZ\nJobhFs+sLf0HAFWJbtLk1gvH3DMz2GJKDBSoFAK5hYsVLq4NpybynhvSi09ZlQEc\nRPCCe0YaDvecScQbRHIsOVseHVn5M3Zi9AviTwx2qgkEw7nWeDNMURFDd4QWquAZ\nI7nEOLO43fkxu8ohiY+3EddXW39VY61c7KQAx5ITRil8JMa+jLeE0IIGXSmN0kPj\nuQIDAQAB\n-----END PUBLIC KEY-----"}, "followers": "http://bw.builder.hyperborea.org/user/bookwyrm.instance.actor/followers", "following": "http://bw.builder.hyperborea.org/user/bookwyrm.instance.actor/following", "outbox": "http://bw.builder.hyperborea.org/user/bookwyrm.instance.actor/outbox", "endpoints": {"sharedInbox": "http://bw.builder.hyperborea.org/inbox"}, "icon": {}, "bookwyrmUser": true, "manuallyApprovesFollowers": false, "discoverable": false, "hideFollows": false, "@context": ["https://www.w3.org/ns/activitystreams", "https://w3id.org/security/v1", {"manuallyApprovesFollowers": "as:manuallyApprovesFollowers", "schema": "http://schema.org#", "PropertyValue": "schema:PropertyValue", "value": "schema:value"}]}

http://bw.builder.hyperborea.org/user/bookwyrm.instance.actor#main-key - What GTS is looking fow http://bw.builder.hyperborea.org/user/bookwyrm.instance.actor/#main-key - What the JSON identifies the key as.

If GTS is looking for the exact ID in the JSON, that's probably where the communication is breaking down.

I'm not familiar enough with the spec to say which side should be normalizing this, though I'd tend to go with the robustness principle and say that both should.

hughrun commented 1 year ago

@kvibber is right that we're using mismatched key ids and should fix it. However, I believe the primary problem is caused by this issue: https://github.com/bookwyrm-social/bookwyrm/issues/2801

I've got an almost-working fix, but unfortunately whilst testing that, another problem with mentions has surfaced so I need to work out what's going on with that.

bahuma20 commented 1 year ago

Seeing the same issue when trying to POST a Follow activity to a users inbox from Mastodon 4.1.2 as well as my own server that uses activitypub-express 4.2.1.

Example POST https://books.theunseen.city/user/bahuma20/inbox:

{
  "@context": "https://www.w3.org/ns/activitystreams",
  "id": "https://mastodon.local/13218191-0689-4d45-8f3e-d9027c812959",
  "type": "Follow",
  "actor": "https://mastodon.local/users/bahuma20",
  "object": "https://books.theunseen.city/user/bahuma20"
}

Signature header:

keyId="https://mastodon.local/users/bahuma20#main-key",algorithm="rsa-sha256",headers="(request-target) host date digest content-type",signature="vjLkSuw9qYdz3DB2X/Am7uVPyFERQm4Qn+sJFPef0o0y5YmffSUUtQDCZxW1500JNqXxqNmU63QpXrFGIRM4pfASzWb5F9D9P18Xtm9Xt5uL0GVHH7y9u5w9EYBIdZUAOkYCehoiv4g1UBI06XCPXAqukd28Wk7DdJTL9Y7v5jBR8Vn4tYMPqH5LEU8W0XYX6ZozlKF+mT4UYSL36URyd+06HiJEAu3Ns+Xv4tao9SAWFpQgE6xHLcpaQp1MtQfibwWeC5TN8t1g6UwJf92ES3jmnPWY/kwcQSIPScZsuWgyKvvqX3Lt1r7pJ+T6PcHzwl9fMCJD3XDdPG6Snc9akw=="