retrieve feeds individually

[sknebel]

In my traditional reader, I regularly access individual feeds in folders ("channels").

API-wise, this could be implementing by giving feeds a UID as well and allowing those as parameters in the action=timeline requests (and potentially others where it makes sense).

[EdwardHinkle]

I can see the benefit to this as well. While not used by every reader, I can definitely see a category of readers that would appreciate the ability to use it in this way.

[swentel]

I'm getting this use case more and more myself, usually in two situations:

• I'm in a channel, I see a post from a feed mixed with many others, but would like to see all posts directly from a certain feed.
• I would like to 'search' for all posts from a feed. If I know the feed url, I could enter the URL, or partially.

There's some brainstorming at https://indieweb.org/Microsub-spec#Searching_for_Content - if the value of 'query' would be a full url, the backend could decide to only match on that. However, that's a bit tricky IMO, so I like the idea of action=timeline, adding 'url' as another param where the value is the feed url: so action=timeline&url=https://aaronparecki.com/primary - Not sure if channel should still be passed on here.

I'm going to test this out with the Drupal IndieWeb module and Indigenous, making it optional in Indigenous via Settings.

[swentel]

I implemented this in Drupal and Indigenous. It's a setting in Indigenous to enable it. Clicking on the author avatar opens all posts by this author/feed.

From a spec perspective:

• every microsub item gets a _source property, which contains the id of the feed. This is the internal id of the feed on the microsub server. Makes it a bit easier than using url imo.
• the microsub server accepts a GET request where action is timeline and a parameter source

Feedback welcome!

I'm going to add a (global) search too, but that belongs into another issue.

[aaronpk]

That's awesome, I've been wanting this too. How are you handling this in the UI?

[EdwardHinkle]

Right now he added where when you tap on a user’s avatar, it takes you to a view that shows the posts from their feed

[aaronpk]

Does it show what feed you're looking at somehow? what is it labeled? How do you get out of this view?

[EdwardHinkle]

Ohhhh, I see. Haha! Good question. @swentel? 🙂

[swentel]

Here's a quick (pixelated) video :)

• it opens a new activity and I pass on the author name for the individual feed. In case it's just a standard feed, you'd get https://example.com/rss

• it also shows how the search in a channel works, it passes on the search query string to a new activity (feedback welcome on https://github.com/indieweb/microsub/issues/33 for that)

[aaronpk]

I am looking at implementing this in Aperture+Monocle now.

I have a few multi-author feeds in my account, such as my Micro.blog timeline, Instagram timeline, and some Twitter searches. For those cases, the "source" feed is always the same. This implementation of using a "source" ID will end up treating these as a single source, which I suspect is not the end experience I'd expect. I think I'm going to implement this anyway to try it out, but that's something to keep in mind.

[aaronpk]

Since Monocle is completely stateless, I needed to add some information about the source into the response in order for Monocle to be able to display that. Here's a sample of my request and response now:

/microsub?action=timeline&channel=X&source=Y

{
  "source": {
    "name": "Name Here"
  },
  "items": [
    ...
  ],
  "paging": {
    ...
  }
}

(In the case of Aperture, the source name is actually a version of the URL, since I don't actually store any data about the names of sources right now. Eventually I'd like to have a better display name and also probably an icon.)

[swentel]

I also have a twitter feed where indeed the source id is the same. Indigenous/drupal can search as well, so instead of triggering individual feed, it could trigger a search on the author name. However, not sure if I can pass on that into the feed response to distinct between the 2 cases unless we introduce a new property on source level here, or let clients configure this. I would opt for the first option though.

[aaronpk]

This is live in Monocle now! I ran into a few edge cases that I'm not clear on how I want to solve. Mainly around reposts and likes. Also when there is no author info, currently Monocle doesn't show anything in the space the author normally appears, so there's nothing to link to the source there. (This has been an annoyance with Monocle for a while but I'm still not sure how I want to solve it).

[mblaney]

I'd be interested in trying this out with my own server, @aaronpk does Aperture trigger something in Monocle to display extra UI for this? Also @swentel I would be happy to test Indigenous for Android with my server but I can't log in to the app... got any error logs on your end that could help? I get told it doesn't find the required endpoints when given https://unicyclic.com/mal

[swentel]

@mblaney I have not pushed a release yet with these changes, will be for the weekend, I'll ping you here again when that is done, and have a quick look why it might not find the endpoints.

[aaronpk]

@mblaney yes, Monocle looks for the _source property and if it finds it, it will turn the author name and photo into a link to view posts from this source. This way there is no change in Monocle for servers that don't support this property.

[mblaney]

cool, so is _source just the id? or is #13 getting factored in here too?

[kevinmarks]

if there is only one feed in a channel, might want to disable this as it is a bit confusing

[aaronpk]

@mblaney right now _source is just a string. Maybe it'd be better to expand that right now to include that h-card object described in #13?

Combining both proposals would basically mean the entry would look like this:

{
  "type": "entry",
  "published": "2019-04-16T22:57:03-04:00",
  ...
  "_id": "1111111",
  "_source": {
    "_id": "2222222",
    "type": "feed",
    "url": "https://example.com/feed",
    "author": {
      "type": "card",
      "url": "https://example.com/",
      "name": "Example Feed",
      "photo": "https://example.com/photo.jpg"
    }
  }
}

To keep things in the spirit of Microsub, I wouldn't want to have _source vary between a string and object type, which would be the other way to do it.

But also I don't want to require all the rich feed info in the response, so I think that means either _id or type/url/author would be required for the _source value. If there is no _id then the client wouldn't be able to link to the query to return items just in that feed.

What do you think @swentel?

[mblaney]

I still like the idea I mentioned in #13 about fetching the feed information separately. that way you could keep _source in the item as a string, and use it to match an entry returned from a new feed level request.

[swentel]

@aaronpk I like the idea of 13, however I currently do not store that information at server level when subscribing to a feed, only the URL. I have the advantage in android I can pass on info between screens (even though it's just the name/url at the moment) so the current implementation is enough for me.

However, it's clear I need start storing additional info so my server can work with monocle too for individual feeds because it's stateless.

I do prefer just sending only _source id at the moment in the individual source item though as usually you have info already from the author as well (except that the feed url might be different than the homepage url).

Sending the full h-card in the individual response probably makes more sense as (especially stateless) clients can show even more information directly about the author.

That's my current feeling, back at the keyboard this weekend, so I'll ping back to confirm :)

[swentel]

Pushed a new release of Indigenous. The Drupal microsub server now also returns the 'source' property with a 'name' property in the individual response, works great with monocle now too.

I'm going to start working on storing feed information on the microsub server so I can start send a better title as I'm now also sending a URL there.

Reading my previous comment again: I like the current implementation at the moment. But as already said previously, it probably makes sense to send more info in the individual response so clients have more freedom in the presentation: e.g. drop all the author avatar/links in every post item for and have a dedicated single widget that shows the author/feed information.

@mblaney indigenous needs a couple of endpoints which it can't find on https://unicyclic.com/mal/

• authorization_endpoint and token_endpoint: to authenticate and exchange tokens
• microsub or micropub: to connect to a reader and/or post to a site

[mblaney]

thanks @swentel all the endpoints are there, I'm wondering if you have more details in your logs about the problem?

[swentel]

@mblaney Oh I see why, the endpoints are in the header response, Indigenous doesn't recognize those yet, my bad. Opened an issue at https://github.com/swentel/indigenous-android/issues/221, will fix that this week!

Apart from that, there are two meta tags before the HTML tag in the response - doesn't affect this bug though, just wanted to let you know :)

<!DOCTYPE html>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width">
<html>

[mblaney]

oops didn't realise they need a parent tag! thanks :-)

[grantcodes]

I have implemented this in together now as well, using the same idea as everyone else: click on the author and see posts from that source.

Again though I run into the problem of multi author feeds and that interaction just not quite feeling right.

There's 2 ways I can see that potentially being fixed:

1. Make the _source property an object with more details, and clients can check for differences between the author and the source url and make a more intelligent decision about what to show.
2. It's all just magically handled on the server - the server can probably track the source feed and the author and make the _source property a compound of the two, then use that when querying.

There are definitely a lot of negatives and positives with both of those potential options though.

[swentel]

Experimented with Indigenous a bit. When the source url of the post contains twitter.com it now triggers a search, which works great (together with the Drupal microsub server). However, it's only a stopgap, I can't really do anything when a post comes from news.indieweb.org for instance.

More properties in _source is probably the easiest option, with gives the responsibility to clients. The second suggestion sounds way more harder to manage for the server imo.

[swentel]

Some more thoughts: more properties in _source might not be even ideal either. The url for a source might be feedburner.com for instance, so doing a comparison between the post url and source url might trigger false positives.

Which then leads me to a third suggestion: allowing the user to toggle a checkbox which indicates that this is an aggregated feed, and then adding _aggregated: true|false. Ideal? Probably not, but failproof I guess.

[swentel]

I've fixed the 'aggregated feeds' problem by created a compound variable on the drupal side. This works beautifully now in combination with Indigenous which doesn't have any idea about this, woohoow!

[dshanske]

Enough implementations to add to spec.