rssCloud bootcamp

[scripting]

I want everyone to use rssCloud, it's very easy once you have code to crib.

right now I'm working on an OAuth connection to Mastodon, so I can hook FeedLand up to the Fedioverse.

I've heard some people are trying to get new rssCloud implementation working. I started this thread for them to ask questions, and maybe we can help (sure we can).

[colin-walker]

I think I've got it set up on my feed okay — When it gets recreated I be got the ping to rpc.rsscloud.io working and it showed in the log 🤞

What I need to look at now is subscribing to feeds and then receiving the updates in my reader. I'll look at the documentation and fire questions here if I need to.

[scripting]

@colin-walker — great!

[colin-walker]

@scripting Quick question: the feed is definitely okay and I have an endpoint (/notify) responding correctly to the /pleaseNotify challenge. The logs on rpc.rsscloud.io shows that it was notifying me when I made a change (lastbuilddate) ... but ... how is that notification sent so that I can react to it in /reader?

Thanks.

[scripting]

@colin-walker --

The cloud server will call you according to what's in the cloud element in your feed. If you want me to help figure out exactly what that is, you need to include the feed URL. (Update: At this point I misunderstood the question. It was a long day slogging through OAuth code.)

Also, it's better not to direct general questions to me personally, people are less likely to help if they think you want the help from one person in particular. Lots of years of experience with this. ;-)

[colin-walker]

Understood :)

It's my normal feed.

I copied the cloud element from other feeds:

<cloud domain="rpc.rsscloud.io" port="5337" path="/pleaseNotify" registerProcedure="" protocol="http-post"/>

I can register successfully but need to just sort out how to process the notifications.

[scripting]

@colin-walker -- I just re-read your initial question and it's confusing, and that's probably the problem. :-)

There are three actors in this setup.

1. The feed, which has a cloud element that tells a subscriber how to request notification.

2. The CMS that pings the cloud server when the feed updates.

3. The aggregator that gets notified by the cloud server when the feed updates.

I think your question is about #3?

The aggregator has to connect with the cloud server every 24 hours to renew the subscription, otherwise it ages out.

You seem to say that is already set up.

So maybe the only thing you're missing is the ping, #2?

I have code you can look at for examples if you're good with JavaScript.

[scripting]

Example code to look at --

1. For the ping -- daverss package.

2. For renewing subscriptions, River5.

[andysylvester]

@colin-walker, here is another example code I ran earlier this month (using Dave Winer code examples):

https://andysylvester.com/2022/11/09/demo-of-rsscloud-protocol-and-reallysimple-npm-module/

[colin-walker]

Thanks guys. I'm working in PHP.

Let me try to rephrase:

1. The feed has the cloud element (hopefully it's got the correct info)

2. Each time the feed is rebuilt it fires off a ping and the log on rpc.rsscloud.io showed that was successfully received

3. I was able to successfully subscribe via /pleaseNotify and the logs indicated that I should have been notified in response to the pings. (This has all been within a 24 hour period so renewing the subscription is not an issue at present.)

My question is:

• how does the notification happen and what do I need in the reader to receive it?

Is that any clearer?

Thanks again. Sorry if I'm being dumb and missing something.

[scripting]

OK I can answer that.

Here's the data that River5 sends to the cloud server when renewing a subscription.

        var theRequest = {
            url: urlServer,
            followRedirect: true, 
            headers: {Accept: "application/json"},
            method: "POST",
            form: {
                port: port,
                path: path,
                url1: urlFeed,
                protocol: "http-post"
                }
            };

The notice from the cloud server comes to you as an HTTP request on the indicated port and path.

That's it. That's how it calls you.

[andysylvester]

If you look at this function (https://github.com/andysylvester/myStatusDemo/blob/main/rssCloudTestOriginal02.js#L72), your app should be able to respond to a POST from the rssCloud server (if I understand the protocol correctly).

[colin-walker]

Thanks. That's what I was thinking but couldn't get it to work.

I've tweaked it and am now seeing "failed to notify" messages in the log.

What is the format of the http request it sends for the notification?

Does it send a challenge with the notification the same as with pleaseNotify?

[scripting]

According to the docs, it's an HTTP-POST with one value in the body the URL of the feed that updated.

http://walkthrough.rsscloud.co/

[scripting]

The challenge is exactly the same as what you receive as notification.

You can figure all this stuff out with a debugger or writing to the log. Can't you do that in PHP?

[mblaney]

I've implemented rssCloud in PHP. Here's some code if that helps: https://gitlab.com/dobrado/dobrado/-/blob/master/php/cloud.php

[dodyg]

I will work on C# / .NET 7 implementation.

[scripting]

@dodyg -- that's great! just like old times. ;-)

@colin-walker -- just checking in to see how it's going if you can use any help.

[colin-walker]

Got the chance to look at it again today and it's now working. Realised I was making a stupid mistake: I was giving the path as /notify but this auto-redirects to /notify/notify.php and, in doing so, the POST variable was getting reset and the notification failed. Changing to the full path has it working.

Now I just need to cycle through each feed in my reader and send a subscribe once every 24 hours where there is a cloud element. 👍

[scripting]

@colin-walker -- that's the way it goes. i spent four days trying to get a basic Hello World call to work using Mastodon's API.

There was one bit where I guessed wrong about something, and forgot that I did it. Working on this stuff puts you in haze. But it's a nice feeling when you get it figured out.

Let me know when I can tell people about what you did. Proud of your accomplishment. ;-)

[colin-walker]

@scripting Thanks :) I'm going to be blogging about it all when I return to posting properly on Friday. I just need to ensure that /reader updates correctly and the subscriptions are renewed okay and will then be all good to go.

[scripting]

@colin-walker -- okay cool. if you need help, just say the word, and when you're ready please post the url in this thread. ok?

[colin-walker]

Quick question: when subscribing, is it better to send one request with url1, url2, etc. or is it okay to send multiple requests with just one feed? Do multiple requests cause any issues?

[scripting]

@colin-walker -- it's fine to send one request per feed. That's how FeedLand does it. That was something we put in because the WordPress guys wanted it. I guess they deal in much larger sets of feeds.

[colin-walker]

Looks like everything is working \o/

@scripting your new scripting.news posts arrived in my reader without me doing a manual pull.

I set up an email alert to let me know when I received a notification from rsscloud, checked /reader and there they were!

Thanks for all your help.

[scotthansonde]

I have a static blog built with eleventy. I wanted to implement RSSCloud, but a web search with "eleventy" or "Jamstack" came up with nothing. My host supports supports serverless functions, so I wrote one to ping http://rpc.rsscloud.io every time the blog deploys. Now my posts show up immediately on http://feedland.org. 😃 I wrote a quick post with the code.

[colin-walker]

I have announced cloud support here

[scripting]

Colin, I pushed it out via my linkblog.

I will write about this project on Scripting News, I want to point people at the bootcamp.

I also need to create better docs, I don't like the walkthrough -- too chatty. We can do better now.

Right now am working on getting Radio3 to work with Masto. Configuring for all the servers is an order of magnitude more complicated that Twitter, with only one server to work it out with. ;-)

[colin-walker]

Indeed, especially with all the individual terms server admins may place on using them, not that I can imagine anyone really worrying about their API used that way.

As a side note, does the Scripting News nightly feed not ping with updates? I don't receive notifications for it.

[scripting]

@colin-walker -- a few of my many feeds don't ping. wouldn't surprise me if the nightly one didn't. i'll swing around to it when i get a chance. ;-)

[andysylvester]

I am investigating if WordPress.com support for rssCloud works. I think there is a problem, I created a WordPress.com forum post (https://rsscloud4.wordpress.com/2022/12/04/rss-cloud-support-in-wordpress-com-not-working/) to document the issue. I do not think it is a problem with FeedLand, but posting it on this thread for awareness. My next step is to look at the WordPress plugin for RSS Cloud (https://wordpress.org/plugins/rsscloud/) to see if there are issues there.

[andrewshell]

Last I looked into it, WordPress implements its own rssCloud server in the plug-in. I’ll see if I have time this week to look as well.

[andysylvester]

Andrew, thanks for responding. From our previous conversations, I remember that WordPress has its own server. In my test script (URL for the script is in the WP.com post), I was sending the request to that website URL in the RSS cloud element and using the procedure mentioned in the RSS file cloud element. I believe that the plugin has the same problem (did some informal tests a few weeks ago, but did not write anything down). I will post my results when I have them.

[colin-walker]

@andysylvester I was getting the same error (a valid feed was not provided) when trying to subscribe to https://brokenriverbooks.com/feed via my reader (also when testing with Postman) so definitely don't think it's FeedLand.

[scripting]

I wrote a top-level explainer for rssCloud.

http://this.how/rssCloud/

[andrewshell]

I've been doing some testing against https://brokenriverbooks.com/feed and its cloud server.

1. If my aggregator/testing app is on HTTP and I post to https://brokenriverbooks.com/?rsscloud=notify with a specified domain, everything works as expected.
2. If I omit the domain but specify the correct port number, it does not seem to work. Perhaps $_SERVER['REMOTE_ADDR'] in the WordPress plugin isn't correct because of a proxy or something?
3. If I post to http://brokenriverbooks.com/?rsscloud=notify, I get a 301 moved permanently, which I'm not sure if FeedLand handles correctly.

So basically, I can confirm this works:

curl --location --request POST 'https://brokenriverbooks.com/?rsscloud=notify' \
--header 'Content-Type: application/x-www-form-urlencoded' \
--data-urlencode 'domain=test.rsscloud.io' \
--data-urlencode 'port=80' \
--data-urlencode 'path=/feedupdated-s8759' \
--data-urlencode 'registerProcedure=' \
--data-urlencode 'protocol=http-post' \
--data-urlencode 'url1=https://brokenriverbooks.com/feed/'

[andysylvester]

Andrew, have you tried running your testing app on a port other than 80?

[andrewshell]

Let's not pollute this channel by debugging the WordPress plugin. We can continue the conversation over here: https://gist.github.com/andrewshell/8b57f96bf62a1dea9fa1a8137e13951a

[scripting]

Note I finally got the FeedLand server-side implementation of rssCloud buckled down with the help of @andrewshell.

[pzingg]

If anyone is interested, I just pushed an Elixir RSS Cloud server to GitHub. I had found a nice, tested implementation of WebSub, forked it and then gave it the ability to handle the RSS Cloud API as well. (The two APIs are similar, but have different protocols especially on the notification side.)

My next plan is to integrate a chunk of this code into a fork of the Elixir-based Mastodon server, Pleroma, and add <cloud> elements to the feeds produced there.

[scripting]

After getting the rssCloud implementation in FeedLand debugged, with the help of @andrewshell, I thought it was a good time to preserve my understanding of how it works in an example Node.js app that implements the feed reader side of the three-sided network.

This code could use a review by people who have implemented rssCloud and those who are interested.

The code is here in the reallySimple repo under demos.

https://github.com/scripting/reallysimple/tree/main/demos/clouddemo

Hope this is helpful. ;-)