Specifying Hypermedia

[JakeGinnivan]

There are two types, contextual hypermedia and cross cutting.

A few considerations

• Ideally do not want represenations to inherit from a base class (causes issues with plan application/json content)
• Represenations can have nested representations (i.e Beer.Brewery and both the beer and brewery has hypermedia)
• Concepts/api should not be related to the end format (i.e model after hypermedia concepts, not HAL or Collection+Json)

[JakeGinnivan]

An idea I had is that we could have some sort of Hypermedia lookup which is scoped to each web request. This allows us to separate the strategies to discover hypermedia completely from the serialisation side.

It could look something like this:

class RequestHypermedia
{
    void AddLink(object ref, Link link);
    IEnumerable<Link> LinksFor(object ref);
}

The formatter can look up hypermedia from this collection when it is serialising each object.

On the population side we can have multiple strategies. Some that come to mind are:

• Extension method on object, AddLink(this object ref, Link link)
• Attributes, [Link(ref, link)]
• Hypermedia appenders (runs against a type).

[TradeArcher2020]

That is exactly what I had in mind when I put together this POC project the other day:

https://github.com/chrisaswain/WebApi.Hypermedia

Only, I used a custom filter to populate a Hypermedia wrapper object for the DTO returned in the controller action. Then, the formatter could use the metadata provided by the wrapper object along with the DTO itself to serialize the representation to the correct Hypermedia standard.

I tried to just use the Request, but I couldn't figure out how to get the request from inside the converters, so I opted for wrapping the whole DTO in an object that could contain the metadata so that the formatters and converters could always have access to the metadata they need.

[JakeGinnivan]

Maybe we need a simple example (beers?, employees?) which can be expanded to each of the specific issues we want to cover (nested objects/complex properties etc). Can complex properties have hypermedia appended (does that even make sense, or is that only for embedded properties).

[danbarua]

@chrisaswain I like your idea of stuffing hypermedia in the response context, I'll have a play with that and see if that has legs.

[TradeArcher2020]

When I first started thinking about multiple Hypermedia standards, I looked into all of the most popular standards (HAL, Collections+JSON, Siren, and couple more that I don't remember anymore). I think the api developer should have the option to provide Hypermedia metadata for not only the top level DTO, but also for complex properties contained in that DTO. Based on what I saw in the other standards, they support nested Hypermedia data in some form.

[TradeArcher2020]

Yeah, if we are going to try to support multiple hosts like Nancy, Web Api, etc., then we don't want to tie ourselves to using specific pipeline constructs like custom filters. Perhaps putting all of the hypermedia metadata into the request context and associating it with the DTO or properies on the DTO is the most generic approach. Then, the formatter (if it can access the request context) can use the hypermedia metadata when serializing the DTO.

The question is, can the formatter (and possibly even the converters) get access to the current request? I couldn't find a way to do it in the ASP.NET Web Api 2.2 pipeline. It seems like the converters are meant to be context-less. That's why I ended up trying to encapsulate the DTO in an object that contained the Hypermedia metadata, so that the converter had everything it needed without having to have access to the request.

[andlju]

I think this is one of the things that different standards do differently. HAL has the concept of embedded resources (that in themselves can have embedded resources etc). IIRC Collection+JSON doesn't really do complex types at all by default, but there are extensions to it that do. I'm pretty sure I've seen other standards where links can be sprinkled all over the place.

This may be a tough one to reconcile completely. For HAL I guess we could create an embedded resource whenever someone attaches a link to something other than the root object, but that could make for some rather ugly documents. Another way would be to always have the user specify what parts of a complex document that should be considered an embedded doc and only allow links on those. This, however, may limit other more flexible standards. Not that I can think of one now, but I'm sure I've seen media-types where you can add links at any level in a document.

[wis3guy]

I know my approach is biased by the work i did on the hypermedia appenders, but what you describe is exactly what i was trying to accomplish in my sandbox project (see te diagram and link to the repo in the goals issue). I also stranded in finding a way to pass the request context to the actual formatters.

Imho having links in embedded resources which have their own links, is not ugly at all. It is actually proper use of HAL and i use it extensively in my production api's. One thing to note is that embedded resources are an artifact of the hypermedia caching pattern. In fact, the spec states that you should provide links in the parent resource to all of your embedded resources.

In order to mitigate the differences between the various standards, and to allow developers to take advantage of specific features, i still feel that an architecture with format specific intermediate objects is the cleanest approach. That way you have control over how to deal with complex properties for example. Or curies. Or inclusion/exclusion of certain properties. We could provide conventions and attributes to aid this. The problem i ran in trying to do this, is that it will only work one way (server to client)

Op maandag 24 november 2014 heeft Anders Ljusberg notifications@github.com het volgende geschreven:

I think this is one of the things that different standards do differently. HAL has the concept of embedded resources (that in themselves can have embedded resources etc). IIRC Collection+JSON doesn't really do complex types at all by default, but there are extensions to it that do. I'm pretty sure I've seen other standards where links can be sprinkled all over the place.

This may be a tough one to reconcile completely. For HAL I guess we could create an embedded resource whenever someone attaches a link to something other than the root object, but that could make for some rather ugly documents. Another way would be to always have the user specify what parts of a complex document that should be considered an embedded doc and only allow links on those. This, however, may limit other more flexible standards. Not that I can think of one now, but I'm sure I've seen media-types where you can add links at any level in a document.

— Reply to this email directly or view it on GitHub https://github.com/DotNetHypermedia/DotNetHypermedia/issues/2#issuecomment-64238891 .

Ciao, Geoffrey

Geoffrey Braaf | +31655793290 Freelance .NET Software Architect & Passionate Developer | http://wis3guy.net Findsi: find-as-i | http://www.findsi.com

[wis3guy]

@jake i second the need for simple examples to outline our needs.

Op maandag 24 november 2014 heeft Jake Ginnivan notifications@github.com het volgende geschreven:

Maybe we need a simple example (beers?, employees?) which can be expanded to each of the specific issues we want to cover (nested objects/complex properties etc). Can complex properties have hypermedia appended (does that even make sense, or is that only for embedded properties).

— Reply to this email directly or view it on GitHub https://github.com/DotNetHypermedia/DotNetHypermedia/issues/2#issuecomment-64223408 .

Ciao, Geoffrey

Geoffrey Braaf | +31655793290 Freelance .NET Software Architect & Passionate Developer | http://wis3guy.net Findsi: find-as-i | http://www.findsi.com

[TradeArcher2020]

@wis3guy - Regarding the one way (server to client) problem you mention, I don't see that as such a problem. I think the API should specify to the client what it accepts and the extra bloat of hypermedia should not be a part of that. I think it's fine to only send the extra hypermedia data to the client and not accept it coming into the server. It makes the client have to do a little more work, but for performance reasons, I would think that is worth the extra effort.

[wis3guy]

@chris i tend to agree ;-) At the same time, for scenarios where you want to consume a hypermedia driven api, you do need deserialisation. I'm still not sure where i stand on that personally, but i think i could live with receiving those intermediate objects or dto's and map them into domain object in the client somehow (extension methods or fancy mappers). See also my FindsiClient project for example.

Op maandag 24 november 2014 heeft Chris notifications@github.com het volgende geschreven:

@wis3guy https://github.com/wis3guy - Regarding the one way (server to client) problem you mention, I don't see that as such a problem. I think the API should specify to the client what it accepts and the extra bloat of hypermedia should not be a part of that. I think it's fine to only send the extra hypermedia data to the client and not accept it coming into the server. It makes the client have to do a little more work, but for performance reasons, I would think that is worth the extra effort.

— Reply to this email directly or view it on GitHub https://github.com/DotNetHypermedia/DotNetHypermedia/issues/2#issuecomment-64252171 .

Ciao, Geoffrey

Geoffrey Braaf | +31655793290 Freelance .NET Software Architect & Passionate Developer | http://wis3guy.net Findsi: find-as-i | http://www.findsi.com

[andlju]

@wis3guy Oh I totally agree about embedded documents being necessary and definitely not ugly. But I think it should be an explicit choice to create an embedded document, it shouldn't be done automatically behind the scenes just because someone attached a link to an object somewhere.

[wis3guy]

@anders ok, then we agee ;-)

On Monday, November 24, 2014, Anders Ljusberg notifications@github.com wrote:

@wis3guy https://github.com/wis3guy Oh I totally agree about embedded documents being necessary and definitely not ugly. But I think it should be an explicit choice to create an embedded document, it shouldn't be done automatically behind the scenes just because someone attached a link to an object somewhere.

— Reply to this email directly or view it on GitHub https://github.com/DotNetHypermedia/DotNetHypermedia/issues/2#issuecomment-64254185 .

Ciao, Geoffrey

Geoffrey Braaf | +31655793290 Freelance .NET Software Architect & Passionate Developer | http://wis3guy.net Findsi: find-as-i | http://www.findsi.com

[JakeGinnivan]

If we go the extension method approach formatters (which will be different projects) could take a dependency on DotNetHypermedia.EmbeddedItems which exposes another extension method in the DotNetHypermedia namespace.

Then formatters like Hal can take a dependency on that.

[anders]

@wis3guy think you meant @andlju, not me

[AlexZeitler]

@JakeGinnivan I'll rewrite my approach to use Beers and then push it. In general I second that we should not require to implement some Interfaces or derive from base classes in the DTOs. Also default MediaTypeFormatters (in Web API World) for JSON/XML should create plain XML/JSON without HyperMedia related properties.

[AlexZeitler]

I think what also should be possible is the ability to add Links for different base URIs. Speaking of the beer scenario there could be a host providing the beer types and it is only "referenced" by the "internal" host / API providing the beer / brewery stuff.

[danbarua]

The question is, can the formatter (and possibly even the converters) get access to the current request? I couldn't find a way to do it in the ASP.NET Web Api 2.2 pipeline.

We do this in Nancy.Hal:

config.For<PetOwner>().
  Links(model => new Link("rel1", "/dynamic/{name}").CreateLink(model)).
  Links((model, ctx) => new Link("rel2", "/dynamic/{name}/{operation}")
    .CreateLink(model, ctx.Request.Query)); // <- passes query string parameters into link template

It's a really nice feature to have, the canonical use case I can think of (that I use it for) is creating paging links for paging through a collection filtered by a query string.

[AlexZeitler]

In Web API you access the current request in the GetPerRequestFormatterInstance of the MediaTypeFormatter implementation. That's how we're generating the Href of the C+J collection.

GetPerRequestFormatterInstance is called before WriteToStreamAsync.

[danbarua]

Cool, and in ServiceStack you get access to the RequestContext too.

[wis3guy]

Great! We could simply create additional extension methods that take a MediaAppender class instance, rather than a lambda, allowing for separate classes, which in turn could have their own dependencies (not available in the config). That assumes we call config at application start only, right?

If so then how would we deal with request/response specific links? F.ex. If this is the second resource you hit, i want to add some additional convenience links given your behaviour pattern.

In case we would use appender classes (like i do) we would need access to an IoC container at the point in the code where you do the configuration.

Also, what if we want to use transient dependencies?

Just some thoughts/concerns ...

Ciao, Geoffrey

Geoffrey Braaf | +31655793290 Freelance .NET Software Architect & Passionate Developer | http://wis3guy.net Findsi: find-as-i | http://www.findsi.com

On Tue, Nov 25, 2014 at 10:25 AM, Dan Barua notifications@github.com wrote:

Cool, and in ServiceStack you get access to the RequestContext too.

— Reply to this email directly or view it on GitHub https://github.com/DotNetHypermedia/DotNetHypermedia/issues/2#issuecomment-64330822 .

[danbarua]

As @glennblock suggested both approaches are valid though he prefers the appender class approach:

My reasoning for this is Hypermedia is not a cross-cutting concern, it is very real business logic of the system.

Treating hm configuration as a cross cutting concern is easier to get up and running with (I don't have to add more classes to my project) but on the other hand, using MediaAppender/Writer classes gives me full control (I can have dependencies injected in allowing me to look up information from a datastore, or some per-request service for example).

[AlexZeitler]

I second that with a constraint: HM is a business logic concern. But which HM-Type I choose should be a cross-cutting one.

[wis3guy]

Ok, so we are actually saying the same thing? I got confused by the lambda example, which struck me as 'run once'-only. I do not see, how we could call that on a per request base, right?

@AlexZeitler I think we could also do a combination of app-start config for default link configuration and use per-request/adhoc appenders for adaptive behaviour scenario's.

[AlexZeitler]

@wis3guy In the current project we used UriTemplates to provide links that pointed to external systems based on information (like foreign keys) that live in the system that provides the HM-API. The idea is to have link resolvers that can be attached (provider/plug in model via IoC) for arbitrary return types (DTOS). This allows almost runtime extinsibility for new link types.

[wis3guy]

Ok, i see. that sounds a lot like what i attempted here: https://github.com/wis3guy/HalSandbox/blob/master/HalSandbox/HalSandbox/ResourceBuilderTests.cs

Your approach seems much better, because you managed to have it request context dependent. Cool stuff ...

[danbarua]

And a similar approach taken in CollectionJson.Net https://github.com/WebApiContrib/CollectionJson.Net/blob/master/samples/friendapi/Infrastructure/FriendDocumentWriter.cs

[AlexZeitler]

@danbarua We came from CollectionJson.Net but the problem here is that your controller implementation depends on the CJWriter / CJReader. But... what if we had a base HMWriter / HMReader class - dead you mean that?

[danbarua]

Yes that's what I meant, we're looking to find common functionality to put in a HypermediaWriter/Appender class.

[AlexZeitler]

Could make sense as I think it might become tricky if you're thinking of generating paging routes. This could be easily handled in the controller and might be almost impossible in other places. What I have been thinking of is having some base stuff which might be a depedency for the controller (in Web API world) and the concrete de/-serialization (also based on re-usable components) is triggered in the formatter / whatever (depending on the framework being used).

[wis3guy]

@alexander I wonder how that would translate to Nancy and how the 'in-controller-stuff' would relate to the HypermediaFormatter if we do not have some sort of an intermediate representation that can hold the links as well as the actual object details. Also, having to derive your controller from some base class might conflict with other base classes that one would want to use use. Not saying it is therefore not a viable approach, just acting as the devil's advocate ...

Ciao, Geoffrey

Geoffrey Braaf | +31655793290 Freelance .NET Software Architect & Passionate Developer | http://wis3guy.net Findsi: find-as-i | http://www.findsi.com

On Tue, Nov 25, 2014 at 11:38 AM, Alexander Zeitler < notifications@github.com> wrote:

Could make sense as I think it might become tricky if you're thinking of generating paging routes. This could be easily handled in the controller and might be almost impossible . What I have been thinking of is having some base stuff which might be a depedency for the controller (in Web API world) and the concrete de/-serialization is triggered in the formatter / whatever (depending on the framework being used).

— Reply to this email directly or view it on GitHub https://github.com/DotNetHypermedia/DotNetHypermedia/issues/2#issuecomment-64380591 .

[AlexZeitler]

@wis3guy I would not use the CJController base class as shown in https://github.com/WebApiContrib/CollectionJson.Net but I think the HMWriter / HMReader dependencies should be viable in Nancy also?

[AlexZeitler]

Another experience I made during the current project: The devs did not like it having to implement the mappings from there Beer or Friend class to CJ or HM manually using the reader / writer documents. We ended up using AutoMapper which made things easier and more generic (also you have to bootstrap your own AutoMapper engine to not mess AM config with the global one provided by Mapper.CreateMap() et. al.

[AlexZeitler]

In the end we had a few LinkResolvers and a bootstrapper for CJ which added the MediaTypeFormatters to the Web API config. The controllers and DTOs had no dependencies. DTOs for the first implementation had the constraint to only have properties of .NET base types like string, int etc.

[glennblock]

@AlexZeitler the writers are also the place where the hypermedia itself is injected. Nothing stops you from using AutoMapper within the writer as it just an interface.

[glennblock]

@AlexZeitler The point of having these abstractions is to have good separation of concerns and testability. One can easily inject the writer, mock it out in unit tests etc. It makes the code less brittle (I'd argue) in that the hypermedia generating logic is in a nice compartmented place.

[glennblock]

I definitely agree that using something like Automapper makes sense. The two worlds are not mutually exclusive.

[AlexZeitler]

@GlennBlock As said, the team did not want to use them and based on that decision things evolved into a different direction.

[glennblock]

@JakeGinnivan as far as abstracting way the specifics of each format, I am not sold that this really makes sense as formats can very widely and you end up having to create this potentially leaky abstraction that covers the entire universe of possibilities in a single model.

BTW, there's another project that is attempting to do this and address it across platform: https://github.com/the-hypermedia-project/charter

[JakeGinnivan]

Did not know about the hypermedia project. Will answer #11 separately.

This may not be possible. The reason I tried to kick this off is WebApi.Hal had holes, it had been forked to support nancy with some different approaches which were taken from your C+J library.

It seemed that a heap of people had need for hypermedia in their own context, mine was simply a few presentations so my knowledge bottoms out quite quickly.

At a minimum I hoped the discussions could impact these projects, and we may end up with a .NET HAL implementation project and C+J separate but now they support Nancy, WebAPI or whatever framework. At least we could merge 4 projects into two (if we just take those two combinations). Best scenario we get it working for more scenarios.

It is really good to have your thoughts on this, quite a few of us have experience with a subset of formats and frameworks.

[glennblock]

@AlexZeitler perfectly reasonable. I purposely made the abstractions ultra light as helpers and also for usage in my base CJ controller class.

[glennblock]

@JakeGinnivan I agree that if nothing else, having a way to reuse diff hypermedia libraries across frameworks is a good goal. My CJ library works across frameworks, BUT it requires you to implement some framework specific things like an IResponseProcessor for Nancy etc. It would be nice to not require someone to do this.

[JakeGinnivan]

Makes sense. Maybe more tangible short term goal is to achieve the same with HAL (I think all the HAL libraries are locked to frameworks). Though when someone goes to implement uber for Nancy they go back the drawing board.

Because I have not been using Hypermedia recently I feel pretty confused about where to start with Hypermedia in .NET. Must be a nightmare for someone trying to start playing around with it. Each of the libraries have quite different concepts in the way the API's work and how hypermedia is specified.

Short term as well as the goals, it might be a good idea to put a summary of existing projects and a bit of info. As a starting point for people. Even adding resources like good presentations/books to go look at before diving in.

[glennblock]

Here's one example: https://github.com/webapibook/issuetracker

[wis3guy]

@glennblock Great sample!

[wis3guy]

@JakeGinnivan What if we focus first on rewriting WebApi.HAL to follow glenn's CJ library approach and then either fork/extend his issue tracker sample? At least we achieve some significant improvement and at the same time we learn about the commonalities/differences of the two formats.

[wis3guy]

@JakeGinnivan let us also follow and learn from this repo https://github.com/the-hypermedia-project/charter They are actually also trying to find commonalities between formats (#1 on their list). It seems they also aim to use intermediate representations ... going to read up.

[TradeArcher2020]

Just discovered this new standard:

http://jsonapi.org/

[glennblock]

@wis3guy yeah the concern I have with this approach is ending up with leaky abstractions as hypermedia formats vary significantly. As you said, the project has a goal of abstracting out any specific format and providing a generic model for expressing hypermedia message components that can the be rendered to a specific format.