Enhancement: Provide recommended feeds for user

[vincode-io]

I think it would be useful to provide recommended feeds for users as an incentive to upload their OPML files. With the existing data in the database we could have a query that gets related feeds, by subscription count, excluding already subscribed feeds.

We would do this by finding other users who have subscribed to the same feed as the user and providing their feeds as recommendations. We would then exclude already subscribed feeds. After that, pair down the result by the top 20 scoring feeds.

Here is an SQL query that demonstrates the concept:

select distinct f1.countSubs, f1.title, f1.feedUrl
from subscriptions s1,  subscriptions s2, subscriptions s3, feeds f1 
where s1.feedUrl = s2.feedUrl
and s1.username = 'brentsimmons'
and s1.username != s2.username
and s2.username = s3.username
and s3.feedUrl = f1.feedUrl
and s3.feedUrl not in (select feedUrl from subscriptions where username = 'brentsimmons')
order by f1.countSubs DESC, f1.title
limit 20

Other criteria would be interesting to add to the database as well. Perhaps a score that determines feed freshness (the last time that a person posted). There might be even more criteria that could be added in the future to improve recommendations.

Ideally the recommended feeds list would work like other lists on feedBase. It would display on the page, allow subscription, and provide an OPML that could be imported into RSS readers.

[vincode-io]

Here's my motivation for asking about this. I wrote a small application that consumes feed list OPML files and preview feeds. It also provides an easy way for users to subscribe to those feeds. It is called Feed Compass. It is actually the brainchild of @brentsimmons (I just implemented it) and we've discussed integration with both feedBase and NetNewsWire for it.

Here's something we came up with:

+----------------+
|    feedBase    |
+----------------+
  |      |     ^
  |      |     |
Hotlist  |     |
  |      |     |
  |  Suggested |
  |      |     |
  |      |     |
  |      |   Subs
  V      V     |          
+----------------+               +----------------+
|  Feed Compass  |  <-- Subs --- |   NetNewWire   |
+----------------+               +----------------+

All data exchanged would be in a feed OPML format. Feed Compass would have the following responsibilities.

• Display the feedBase Hotlist (already done)
• Display the feedBase Suggestions (if implemented)
• Provide the ability to upload OPML subscriptions from RSS Readers
• Directly ask NetNewWire for the user OPML subscription list to upload

Basically what I think the Suggestions OPML file will provide is incentive for users of Feed Compass to upload their OPML files. It remains to be seen how successful Feed Compass will be. It might not add much to the number of feedBase users, but I think it will be fun to try.

[scripting]

Maurice -- I posted a comment here yesterday, or I thought I did. It doesn't seem to be here. So I'll write it again, with more. ;-)

1. I'm not sure what Feed Compass does that feedBase doesn't. It seems at first it's that you can present a preview of the content in the feed. I have plenty of code that does this, in other products, it probably should be factored into a NPM package that does it, and perhaps include it in feedBase. Or maybe it should be an external app that's linked to from within feedBase? I do that for viewing XML files (since the browser mangles them so much).

2. I think the key feature every feed reader should support is dynamic OPML. That is, in addition importing an OPML file, it should allow the user to subscribe to it. Periodically it reads the subscribed-to OPML file and adds or removes from the internal list of feeds the user is following. I've been advocating this for many years, and it's always been fully supported in my feed reader software since the early 2000s. I made an attempt to define the concept of a "reading list" in this post in 2009. I just wrote a new post about it ten years later.

3. I like the idea of feedBase doing recommendations. I tried to implement it myself a few months ago, but my SQL skills aren't up to it. When I swing around to this, I will look at your post here for a place to start, and hopefully you will be able to coach me in its implementation. I was also thinking of adding a feature to feedBase to allow it to run user-defined queries, but I have no idea how to make that secure (i.e. to disable modification to the database, just queries).

4. I don't like to host design discussions like this in GitHub threads, because the work I propose here isn't for any specific product, such as feedBase and NetNewsWire, rather the offers are open to all developers. I much prefer doing this on blogs, or perhaps in a thread in a different repository (this one is focused on a specific product of mine). I'm going to give it some thought.

[vincode-io]

1. If you added feed previews to feedBase and the ability to subscribe to external OPML files to feedBase, then Feed Compass would only be a macOS version of this. I think that would be awesome if done. Feed Compass being a native macOS app is going to reach a much smaller audience than feedBase can.

2. Agree totally. Think of Feed Compass as an RSS Reader add-on or companion app. It does do OPML subscription. You can even subscribe to arbitrary OPML files under the File menu. It would be better if the individual RSS readers added it themselves tho.

3. I would love to work with you on this. My node.js skills aren't that advanced, but I have done a ton of server side, web, and database development in a past life. Ad-hoc queries are always challenging. Allowing them to be run against an operational datastore is controversial because of how it can impact the performance of the operational system (the website). Generally I think you want to have a strict subset of criteria that can be run by the user to mitigate the performance and security concerns. We can discuss more later.

4. I did put up a blog post and tweeted it at you, but it looks like my AWS instance froze up on me last night. I'm not sure if you got to read it. https://vincode.io/blog/2019/03/24/algorithmic-blog-lists/ I'm cool with opening this up to a larger audience. Let's continue the conversation on the open web.

[scripting]

Sorry to be so long in responding, I am in the middle of moving from the city to the country. Rarely has my life been more disrupted.

1. I will add feed previews to feedBase. It already has the ability to subscribe to a feed with its URL but the feature is really well-hidden (I will fix that).

2. We're at an impasse on recommendations. I didn't understand what you asked me to do. I think you think I know more about SQL databases than I do. I won't be able to do this work until I can re-focus, and that's going to be a while.

3. I'll look at your blog post now.

[scripting]

This is where the Add-Feed command is --

[vincode-io]

I think recommendations are ready to go in their current state. I’d like to make them run faster, but they aren’t unacceptably slow in my opinion.

I would love it if you implemented the UI for recommendations. We can go back and do the performance tuning when you have more time free.

[scripting]

It is too slow for the kind of UI we use for other lists.

It has to be faster or I'll have to come up with another UI.

On Thu, Apr 4, 2019 at 12:24 PM Maurice Parker notifications@github.com wrote:

I think recommendations are ready to go in their current state. I’d like to make them run faster, but they aren’t unacceptably slow in my opinion.

I would love it if you implemented the UI for recommendations. We can go back and do the performance tuning when you have more time free.

— You are receiving this because you modified the open/close state. Reply to this email directly, view it on GitHub https://github.com/scripting/feedBase/issues/42#issuecomment-479967889, or mute the thread https://github.com/notifications/unsubscribe-auth/ABm9O9DzanfCJuaDb0QHmPgRNnx-CSYsks5vdicngaJpZM4cACOP .

[vincode-io]

I say we make it go faster then. I'm retired and just do open source programming, so I'm available when ever you are.

[scripting]

Excellent. That's the best answer. ;-)

On Thu, Apr 4, 2019 at 3:52 PM Maurice Parker notifications@github.com wrote:

I say we make it go faster then. I'm retired and just do open source programming, so I'm available when ever you are.

— You are receiving this because you modified the open/close state. Reply to this email directly, view it on GitHub https://github.com/scripting/feedBase/issues/42#issuecomment-480039309, or mute the thread https://github.com/notifications/unsubscribe-auth/ABm9O9FoPRwQSLnVAbORX1T0xPGFbdceks5vdlfegaJpZM4cACOP .

[scripting]

What did you have in mind for a feed viewer?

[vincode-io]

For the feed previews in feedBase? If so, I didn't have anything in mind.

[scripting]

Yes, I meant in feedBase. How does it work in your product?

[vincode-io]

Here's what I do.

1) Fetch the RSS Feed via an HTTP Request 2) Parse the feed into an object graph using RSParser (this is a Objective-C/Swift library from Brent Simmons) 3) I display it in an NSTableView

It looks like this when I'm done:

[scripting]

Thanks for the explanation. I have a good idea of how I'll do it.

In the meantime, there's a new "Add Feed" button when you view your own subs on feedBase.

Here's the write-up..

https://github.com/scripting/feedBase/blob/master/changenotes.md

Dave

On Fri, Apr 5, 2019 at 6:19 PM Maurice Parker notifications@github.com wrote:

Here's what I do.

1. Fetch the RSS Feed via an HTTP Request
2. Parse the feed into an object graph using RSParser (this is a Objective-C/Swift library from Brent Simmons)
3. I display it in an NSTableView

It looks like this when I'm done:

[image: Screen Shot 2019-04-05 at 5 16 31 PM] https://user-images.githubusercontent.com/16448027/55659355-efcf6b80-57c6-11e9-9ee7-1d0a6fc9979b.png

— You are receiving this because you modified the open/close state. Reply to this email directly, view it on GitHub https://github.com/scripting/feedBase/issues/42#issuecomment-480439942, or mute the thread https://github.com/notifications/unsubscribe-auth/ABm9Ox3bufoAZqgY97A19V5tH5kQSyirks5vd8vxgaJpZM4cACOP .

[scripting]

Demo of the new feature --

https://www.youtube.com/watch?v=VvjHTfrlh8A&feature=youtu.be

Dave

[vincode-io]

That looks pretty cool. One thing I found useful in the previews was the published date of the article. It makes it possible to see how "fresh" a feed is at a glance. It is one of the key deciding factors as to if I subscribe to a feed or not.

[scripting]

Good idea.

On Mon, Apr 8, 2019 at 12:47 PM Maurice Parker notifications@github.com wrote:

That looks pretty cool. One thing I found useful in the previews was the published date of the article. It makes it possible to see how "fresh" a feed is at a glance. It is one of the key deciding factors as to if I subscribe to a feed or not.

— You are receiving this because you modified the open/close state. Reply to this email directly, view it on GitHub https://github.com/scripting/feedBase/issues/42#issuecomment-480910779, or mute the thread https://github.com/notifications/unsubscribe-auth/ABm9O-RImCXcw5TusA5vsjcXICXHLUWKks5ve3KRgaJpZM4cACOP .

[scripting]

The feature is in. You can see it on this page, for example.

http://feedbase.io/?feedurl=http%3A%2F%2Fscripting.com%2Frss.xml

Dave

[vincode-io]

That looks really good! I think it adds a lot of utility to the feed view page.