400 on follow from streams

[macgirvin]

Following tags used to work from streams. Now the Follow activity is consistently bounced with a 400 error. Activity follows:

{
  "@context":[
    "https://www.w3.org/ns/activitystreams",
    "https://w3id.org/security/v1",
    {
      "nomad":"https://fediversity.site/apschema#",
      "toot":"http://joinmastodon.org/ns#",
      "litepub":"http://litepub.social/ns#",
      "sm":"http://smithereen.software/ns#",
      "manuallyApprovesFollowers":"as:manuallyApprovesFollowers",
      "oauthRegistrationEndpoint":"litepub:oauthRegistrationEndpoint",
      "sensitive":"as:sensitive",
      "movedTo":"as:movedTo",
      "alsoKnownAs":"as:alsoKnownAs",
      "EmojiReact":"as:EmojiReact",
      "discoverable":"toot:discoverable",
      "indexable":"toot:indexable",
      "wall":"sm:wall",
      "capabilities":"litepub:capabilities",
      "acceptsJoins":"litepub:acceptsJoins",
      "nomadicLocations":"nomad:nomadicLocations",
      "Hashtag":"as:Hashtag",
      "canReply":"toot:canReply",
      "approval":"toot:approval",
      "isContainedConversation":"nomad:isContainedConversation",
      "conversation":"nomad:conversation",
      "commentPolicy":"nomad:commentPolicy",
      "eventRepeat":"nomad:eventRepeat",
      "emojiReaction":"nomad:emojiReaction",
      "expires":"nomad:expires",
      "directMessage":"nomad:directMessage",
      "Category":"nomad:Category",
      "replyTo":"nomad:replyTo",
      "copiedTo":"nomad:copiedTo",
      "canSearch":"nomad:canSearch",
      "searchContent":"nomad:searchContent",
      "searchTags":"nomad:searchTags"
    }
  ],
  "id":"https://fediversity.site/follow/838",
  "type":"Follow",
  "actor":"https://fediversity.site/channel/test",
  "object":"https://relay.fedi.buzz/tag/streams",
  "to":[
    "https://relay.fedi.buzz/tag/streams"
  ],
  "cc":[

  ],
  "signature":{
    "type":"RsaSignature2017",
    "nonce":"3775498f4982f1238b1f935d0bbda0fa46e9bf80bc27292f73165622549ce952",
    "creator":"https://fediversity.site/channel/test",
    "created":"2023-10-27T11:21:56Z",
    "signatureValue":"tiocmBXyF8vXy+FHhxLtyVPoAjl2PYxTk9mlkiXQqHsHL+XUqo2rzvZ4t+YkdvvabhiNQye00c7rsCdIUEd5LENGdUMqxPoZwMc+t201K6glybBklQ1wKr/Src+hY60MiF3GFI93l44sFZeND7/xaRL9B/8YVKRjG3D7wulbGQJKiwh/N094YcDlldkTuSkunNz0V4pKrPv4coeMUrY/SogHIRhcFSKBE+yYaHxd51+7hxBO5Cux5UHyNp6fBwbm+V6KR+17dHxo6DS500qi6GRmnMlvvi7R3TE373TnItfs1lUUMbo3MBnopR3VgwuFrMHKK02LXC9JFMyjysk6M0p6+DJsmg8JhdcNUO3hi06c1GoacNpSGtgVQ3MjRa3YHNqG5Ev5GYJtLUJqqN+9ntZKdUK7XqRofSutOU8+dF/tC8DICtN32o4ElNGZikXa0T4gSCkRignMqgQ6pbZJKjbCiaUYFyaTnbpbBVhgNknYt9OejBndNZX7t1eOIJNuwBOdK7waFyxFcXPrN0O9lF/yQ9ycPlbpE2dnePK/uBPelaS/UbRMDpTuaQa9qabDEJ5b5K+Gw+LtN7FdKZinIjllZ5C81nf3Dc3K3BYbDL4WEfDTGzj3fTIM7s5LkEFSaJJGAlY7DSOZTG5Wc5Y5kpZuvkcGhepby6iPRIOCRck="
  }
}

[astro]

Is there any easy method with which I can reproduce the issue? Can you give me an account on a "streams" instance?

[macgirvin]

Interesting - I was just making sure my "internet-facing" test server was up to date to give you an account, but it's working fine from there. Yet it is consistently rejected from fediversity.site - which is at the same git commit. So now I wonder if my instance is blocked.

[astro]

FediBuzz blocks domains using https://github.com/gardenfence/blocklist but fediversity ain't in it. The relay does not filter followers.

[macgirvin]

OK, I'm stumped. I've temporarily turned on registration-with-approval for fediversity.site. If you reply here with your email domain (or email me mike@macgirvin.com) I'll keep an eye out for it and approve. I'm on Australia time so apologies if this takes a day or so to happen.

Here are the basic instructions for following somebody....

Once in, you'll be asked to create a channel (your fediverse identity) and then go to your Connections (top menu or hamburger menu) and there will be a widget on the top left (assuming desktop) to add a connection. We accept webfinger or URL. URL seems to be more reliable for the buzzrelay for some as yet unknown reason.

If you go back to /connections and see a red dot on the avatar, the connection hasn't been accepted yet. This sometimes takes a minute, but if it's still red after that, there was either an error returned from the follow or it was rejected.

Then if you Edit the connection (/connedit/xxx), there's a link at the top of the page called 'Connection Tools'. From there you can delete the connection and try again if you need to do so.

[astro]

@macgirvin Thank you, I signed up using astro@spaceboyz.net

[macgirvin]

You've been approved.

[macgirvin]

Your DM went to somebody else but a screenshot was forwarded to me. It appears our hosting provider blocks curl/wget requests to reduce script kiddie attacks. If the relay relies on curl requests to function, this explains why it wasn't working and you can close this issue. I'm told this may be fixed for fediverse hosting clients but haven't yet verified. And personally, I would probably choose to leave the filter turned on, even though it means we would be unable to connect with some relays and bots. But the issue was brought up by one of our members who is possibly using the same provider so I needed to follow through. Thank you for taking the time to look into this.

[astro]

ActvityPub won't work without those GET requests with header Accept: application/ld+json. Please have them allowed.

[macgirvin]

I appreciate you taking the time to discover the problem. However, It was curl and wget that were blocked to prevent script kiddie hacking. It is a default in one of the Linux security libraries and has nothing to do with the streams repository code. You might wish to change the User-agent to something besides the default to avoid being blocked by security libraries in the future, and/or by other service providers.

And since you brought up the Accept header, please read the ActivityPub spec again. Section 3.2.

ActivityPub will not work without

Accept: application/ld+json; profile="https://www.w3.org/ns/activitystreams"

It is specified as MUST.

This -

Accept: application/ld+json

is only specified as SHOULD. We accept both forms because this is now the 14th fediverse project I've encountered that has gotten it wrong, and I've tired of correcting people. Again, I appreciate all your efforts and please don't take this as a critique. I'm trying to be helpful.

[astro]

The user-agent is buzzrelay/0.1.0 (+https://relay.fedi.buzz)

---

Regarding the MIME type the spec says:

Servers SHOULD interpret a Content-Type or Accept header of application/activity+json as equivalent to application/ld+json; profile="https://www.w3.org/ns/activitystreams" for server-to-server interactions.

I am actually using application/activity+json here. What is your opinion on that?

[macgirvin]

My apologies for getting activity+json and ld+json confused. I was in a hurry. The point I was trying to make is that

Accept: application/ld+json; profile="https://www.w3.org/ns/activitystreams"

is the only header that ActivityPub implementations are required to support.

We recognise application/activity+json and application/ld+json and a few other variations. My only goal here is to figure out why following the relay isn't working from some but not all streams sites so that our members stop complaining.

In any case the curl issue has been reported as resolved by my service provider. You should now be able to use if tor testing. If you are only using curl for testing and not for the actual relay and the actual relay is not using curl and we aren't on the blocklist that the relay is using, that still doesn't explain why follow requests to the relay from some (but not all) of our sites are being rejected by the relay.

[macgirvin]

Have consulted with my service provider and apparently user-agents containing 'fedi.buzz' were being blocked by security policy as there is apparently a content scraper which contains that substring in the user-agent; and the service provider blocks content scrapers by default that do not provide informed consent.

The source of the issue has been identified and following the relay now works correctly. I will close this ticket. Apologies that I was not able to identify the problem source earlier. This wasn't something I could easily ascertain from my end of the connection as it only occurred on inbound requests, which never arrived at our application.

[astro]

Thank you for following up on this problem.