Package proposal: Post type discovery

[paulrobertlloyd]

In the spirit of creating small un-opinionated packages that do one thing well (rather than attempt to build a single monolith package), a candidate for broader collaboration between node-based projects would be one that returns a post type when provided an mf2 object.

Prior art

post-type-discovery / IndieKit

IndieKit consumes an existing npm package post-type-discovery by @prtksxna. This package mostly works, but I have discovered a few issues:

• Note type posts throws Type Error - I need to create a PR to fix this!
• Type Error when mf2 content property only contains html value
• It doesn’t include support for common (non-specced) post types (such as bookmark, checkin, event)
• It can’t easily be extended to support custom post types
• It assumes that an array of mf2 items will be provided

To mitigate against some of these issues, IndieKit patches around them like so:

const getType = require('post-type-discovery');

module.exports = mf2 => {
  // Experimental post types not supported by getType()
  // @see: https://indieweb.org/posts
  if (mf2.properties['bookmark-of']) {
    return 'bookmark';
  }

  if (mf2.properties.checkin) {
    return 'checkin';
  }

  if (mf2.properties.start) {
    return 'event';
  }

  // Create the `items` array getType() expects
  const items = {items: [mf2]};
  return getType(items);
};

Postr

@grantcodes project deals with deriving post type in it’s own function, which can be found here. It also appears to allow adding of new post types, with the addPostType function.

Abode

@EdwardHinkle’s project deals with deriving post type in it’s own function, which can be found here.

Final thoughts

This is one of the key building blocks of a micropub endpoint – and follows an existing spec – but also one of the most error prone as it needs to consume mf2 which may be malformed or not include expected properties.

It would be nice if there was a single package that dealt with this use case, that was also extensible and adaptable to differing requirements (i.e. accepting an array of mf2 objects, or just a single object).

@indieweb/post-type-discovery anyone?

[martymcguire]

In general I support a package for taking a bag of mf2 and returning the result of Post Type Discovery, but I am not sure I think of PTD as a key building block of a micropub endpoint. I'd be skeptical of the assertion that every micropub endpoint implementor should make use of PTD as a way of pigeonholing posts.

In my personal experience with building (and rebuilding) micropub endpoints, I have found the least pain comes from treating everything simply as "a post" with properties as much as possible, and avoiding classifications like "this is a photo".

There are exceptions, of course. For example, in the most recent iteration of my personal site as managed by Hugo. In order to offer custom iCalendar (.ics) files for events, but not other posts, my endpoint detects h-event and stores those posts in content/events/ instead of content/posts/. Beyond that, however, every post is just "a post", which may or may not have properties like checkin, photo, in-reply-to, etc.

Can we capture other example use cases here specifically for Micropub endpoints needing explicit post types?

[grantcodes]

Yep, definitely think this could be its own, new community run package!

So I think my implementation is the most complete based on https://indieweb.org/post-type-discovery, but I found even that official psuedo code has shortcomings such as missing other h types like h-recipe and h-event, and I've actually seem some people use mp-type in the wild but I think that one is very, very unofficial.

I also don't think this module needs to be extendable per se, it's great for my use case, but if someone wants to add their own post types or whatever I would think it would be pretty simple to do manually.

Technically I think this module would export one function which would do mf2 json -> post type id string. And perhaps an array / object to represent the names of post types (like you see in postr).

---

Also the reply from @martymcguire came in as I was writing this, so RE that:

I think you are right that it probably shouldn't be a base part of a micropub endpoint. But as a module it is very useful for a variety of use cases.

Personally I think it should be a super simple module that people can pull into their own micropub endpoints if they need, and it could be bundled in more complete projects like postr and indiewebkit.

But it is also related to micropub more closely in some of experimental features - like querying by post type and returning what post types are supported in the ?q=config query.

[prtksxna]

I am happy to help with merging pull requests, fixing bugs and pushing new versions out to NPM. Sorry, I've been neglecting that package for a while. I also need to transfer it to me (or indieweb) from the twozeroone org on Github.

[voxpelli]

I agree with @martymcguire, I eg. do no full on post-type discovery but rather just some sniffing: https://github.com/voxpelli/node-format-microformat/blob/5381268dbcdb1aef6a5757758710e4b9f75cbea3/index.js#L329-L337 And that sniffing is optional.

It's a good module and if eg. @prtksxna wants help in maintaining the existing one then lets go for it, but else I think it's a good case for a module where we would gain more from the plurality of there existing multliple ones than from creating a one-and-only one.

The main "issues" here from a collaboration, modularity and interoperability endpoint is:

• What data should the module expect?
• What data should it return?
• Should it be extendable or not?
• Should it be possible to override in cases one doesn't agree with it? (Even when such an override is strictly speaking "wrong" it could be expected in the spirit of plurality)
• How to add new types to it?
• Is the API async or sync?

[grantcodes]

Well I would keep it as simple as possible, so:

• accept mf2 object (not html)
• only return a string of the post type id, or null if unknown
• If it's just a function there is no need for it to be extendable - just run your own function before or after.
• no need for overrides for the same reasoning as above
• also means no need for new types (unless they become popular here)
• sync api - don't think there would be any reason to do async unless you were also requesting html, which I don't think it should do.

My function also doesn't follow the full post type discovery spec exactly, eg. it doesn't check that values are valid urls as it is valid to have nested mf2 objects in place of urls, and maybe people want to somehow like something that doesn't have a valid url, but if the property is defined and not empty it is almost certainly intended to be a like. I also include a bunch of indieweb types that are not in the w3c spec.

But I I am pretty sure @prtksxna module follows the spec exactly. Which is great, but I think there is a good argument to be made to support more post types than are defined in the w3c spec.

[grantcodes]

Just pushed up a version of my implementation of this. Not necessarily saying it should be used but more as a test of a lot of the stuff discussed in #2 & #1

https://github.com/grantcodes/post-type-discovery

[paulrobertlloyd]

Awesome! Will add this as a dependency of IndieKit and see how it fairs. 👍

[prtksxna]

It's a good module and if eg. @prtksxna wants help in maintaining the existing one then lets go for it

Just merged a PR from @paulrobertlloyd's commits (thanks!). I'd be happy to get any help maintaining it.

But I I am pretty sure @prtksxna module follows the spec exactly. Which is great, but I think there is a good argument to be made to support more post types than are defined in the w3c spec.

I am open to adding stuff that is useful even if its not in the spec. Feel free to open an issue.