Open scripting opened 1 year ago
@billstclair — even better than an RSS feed. I’ll explain later today..
@billstclair — I see from your profile you like websockets. Me too! ☘️
Here's the first demo app I want to share with you.
https://github.com/scripting/feedlandSocket
It connects up with the feedland.org server, receiving a realtime stream of new feed items, as FeedLand discovers them.
This is core functionality for the news app that's streaming NYT into the Mastodon server.
I've watched feedland.org's Dev Tools "Network" panel for a little while. It connects to the same WebSocket as the example above, ws://feedland.org:1466/. There appear to be only three messages that pass through the WebSocket, newItem
, updatedItem
, and updatedFeed
. It would surprise me to also see newFeed
, but that doesn't happen very often, and I didn't try to make it happen.
It is indeed a fire hose. Everything comes over the wire. This makes the server code very simple, but I'm wondering how well it scales, whether it's worth keeping a list of desired feeds, defaulting to everything, for each connection, and send only messages for those feeds. Whether the memory of the list of feeds and probing it, for each connection, each time an event happens is a good trade-off to the savings in transmission bandwidth, I don't know.
The feedland.org server knows which feeds those are. Other clients might want to send a subscription list, or a new feed, over the WebSocket. Just thinking out loud. What's there works well. Cool technology.
I'll worry about scaling later if it becomes an issue. It's a tradeoff, right -- complexity and performance on one side, and lots of net traffic on the other. Right now the installed base of FeedLand is tiny and it just isn't an issue. That's pretty much how I work. I tend to over-engineer and provide for situations that never happen. So I compensate for that by choosing not to worry. ☘️
For now think in terms of applications. What can you do with this functionality? I have it piping NYT articles to Mastodon. I plan to open that up as a public service once we turn a few more corners.
BTW there is no newFeed message. The client doesn't care. If the user does, there's a menu command that shows the new feeds to you in chronologic order with checkboxes that makes it dead-simple to subscribe.
Thanks, Dave. Makes sense. It's simple, and it works well.
BTW -- how do I know they're NYT articles? At first I just looked at the URL of the feed the item came from. If the domain was at nytimes.com that was good enough for me.
Then I realize I have a category in my account called NYT that I've put all the NYT feeds in.
And as with everything in FeedLand, I can get at it through a public data structure, in this case an OPML subscription list, the standard data exchange format in rss.land.
Here's the URL for the list.
http://feedland.org/opml?screenname=davewiner&catname=nyt
This mechanism can be used for collections whose URLs don't match.
But I went with the NYT at first because their items are so uniform. No guesswork as to what I'll find in each one. They all have titles and descriptions are short.
We really ought to get this discussion to move somewhere else. I'll link to it so the FeedLand users can see it.
Here's the first item I posted to Mastodon using Radio3.
It's not ready to release yet, it still requires a user interface for configuring it with a specific Mastodon server.
But for now, every link I post through Radio3 will go through Twitter, RSS, the Linkx page on Scripting News and Mastodon.
I chose Radio3 to start with because it was the simplest app to hook up to Masto.
Next question here --
Friends, I need Mastodon-literate folk such as yourselves to try out this new app.
https://github.com/scripting/feedToMasto
It's for Node.js folk, but you don't have to know very much but the basics.
This is an important app -- it's the first one that hooks RSS directly up to Mastodon via the API.
So far no one has validated that it works, so I am reluctant to promote it.
Please help. :-)
Where I'm at.
I have a test Mastodon server up.
I have a bridge server app I can access from a web browser.
I have registered an app with Mastodon.
I have made an oauth/authorize call and get back a code. (Took a lot of trial and error!)
The next step is not working.
I am getting a 404 when I call oauth/token on the same server.
Here's what the call looks like:
https://social.scottfr.ee/oauth/token?grant_type=authorization_code&client_id=xxx&client_secret=xxx&redirect_uri=http://scripting.com/&scope=read+write+follow&code=xxx
These are the docs I'm using.
https://docs.joinmastodon.org/methods/apps/oauth/
I feel like I'm almost there, but missing something.
Any help appreciated.