[RFC] Designing a deserializer

[jrandolf]

Summary^*

A basic design for a deserializer for JSON:API

Motivation^*

The lack of deserialization makes it difficult to traverse JSON:API data. A traversable document deserialized from JSON:API data can be useful for clients and servers alike.

Design Detail^*

interface Options {
 version: string;
}

function deserialize<T extends Dictionary<any>>(document: JSONAPIDocument, options: Options = {}) {
 if (document.version !== options.version) {
  throw new TypeError("Document does not match deserializer version");
 }
 const data = document.data;
 if (Array.isArray(data)) {
  return data.map((datum) => getDatum(datum));
 } else {
  return getDatum(data);
 }
 function getDatum(datum: T) {
  const obj: Dictionary<any> = {};
  obj.id = datum.id;
  if (datum.attributes) {
   Object.assign(obj, datum.attributes);
  }
  if (datum.relationships) {
   for (const [key, relationship] of Object.entries(datum.relationships)) {
    const data = relationship.data;
    if (Array.isArray(data)) {
     object[key] = data.map(datum => getDatum(datum));
    } else {
     obj[key] = [getDatum(data)];
    }
   }
  }
  return obj;
 }
}

Drawbacks^*

There are no links, included resources, and meta.

Rationale and Alternatives^*

This is a simple design. Perhaps only a utility function would be required. An alternative could be designing a "Document Client" which would have methods to traverse the data.

Prior Art^*

Other deserializers do something similar to this, but by doing so lose most of the information they serialize. For example, SeyZ's serializer does this but loses out (correct me if I am wrong) on the same drawbacks.

Unresolved questions^*

The design is meant to be simple and thus give developers the highest amount of motility with respect to their API's response data, but the lack of links and meta is undesirable. We are hoping to get better understanding of the community’s needs through this RFC.

* means required.

[favll]

Here are my two cents on the "included resources" drawback.

We heavily work with included resources in our application and often require a deserialized / denormalized form of resources.

Our deserialize function has a second argument in which it expects a lookup object to retrieve related resources:

const lookup = {
  users: {
    "1": { id: "1", type: "users", attributes: { name: "Jane Austen" }}
  },
  articles: {
    "1": {
      id: "1",
      type: "articles",
      attributes: { title: "Pride and Prejudice" },
      relationships: { author: { data: { id: "1", type: "users" } } }
    }
  }
}

E.g., the following function call will yield a denormalized article object:

const article = deserialize(articleResource, lookup);

{
  title: "Pride and Prejudice",
  author: {
    id: "1",
    name: "Jane Austen"
  }
}

It's the developer's responsibility to ensure lookback contains all necessary resources to resolve all relationships (recursively). Missing resources will result in an exception being thrown. I'm sure this behavior can be improved upon.

We also haven't considered how resolving cyclic relationship graphs would work (although we have toyed around with a getter-based implementation).

[jrandolf]

Using a lookup would be moving toward a query language. At that point, it would be better to use a local data store after adding remote resources. For included resources, I've thought about merely constructing a mapper that takes an object of constructors keyed by a collection name and using those constructors to produce an object keyed by collection names and valued with the constructed object.

The type of the constructor would be as follows

type Constructor<T> = (data: ResourceData, meta: ResourceMeta, links: ResourceLinks) => Promise<T>; // Could also be a destructured object.

data would contain all the attributes, id, and relationships (where relationships are stored as a list [or singleton] of IDs on the object).

Now the primary issue is where to put links and meta. Meta is somewhat apparent, but with the ability to put it almost anywhere, it is a challenge for relationship metadata and the document (and JSON:API) metadata. Declaring a string-keyed attribute directly on the deserialized object isn't a good option, but perhaps using a unique Symbol could be a good option.

[jrandolf]

After much thinking, this package will not include a deserializer mainly because JSON:API clients are meant for this and specialize in specific paradigms (for example, in mobile development).

Moreover, as stated in the current README:

There are many clients readily built to consume JSON:API endpoints (see here). It is highly recommended to use them and only use this for serialization. It would be an anti-pattern not to do so since the problem of serialization and deserialization generally have distinct solutions (think P vs. NP).

For inquisitive developers: To be precise, serialization is optimized by increasing runtime data storage and decreasing computation time (with e.g., caching and stored functions). Deserialization is somewhat dual to serialization; it is increasingly computational with storage proportional to the desired formatting. Perhaps an abstract directed binary tree (ADBT) could be helpful? It turns out the design of JSON:API is not very tree-like (think about the locations the relationships and identifiers can go), so by the time data gets transfigured into an ADBT, we would have finished serializing the data directly.

tl;dr: Serialization and deserialization are different types of actions, therefore must be different packages.

All further discussions will be about a potential typescript deserialization client we can branch to and publish. Contributions are evermore welcome!

[stefanvanherwijnen]

It is not just the clients that need to deserialize JSON:API data. If you'd POST data to a JSON:API server, the server will need to deserialize this data in order to process it. Generally NodeJS is not used as a client so I also don't see a client package for it in the mentioned link.

[jrandolf]

It is not just the clients that need to deserialize JSON:API data. If you'd POST data to a JSON:API server, the server will need to deserialize this data in order to process it. Generally NodeJS is not used as a client so I also don't see a client package for it in the mentioned link.

There is a section in the provided link for Node.js. By the way, every server is a client.

[stefanvanherwijnen]

Thanks, but it looks like those packages are not written in TypeScript (which is basically why one would use ts-japi). I get your point, but I think most people just want to work with JSON:API, and not have to figure out which serializer and which deserializer to use. You are right that deserializing is not the job of a serializer, but they do depend on each other if you want to use JSON:API.

Of course I could implement my own deserializer similar to the example here, but I'd rather use a maintained package. I'm eager to try out ts-japi, but without a deserializer written in TypeScript I can't test any real use cases.

[jrandolf]

Well, if you have an idea for a deserializer, perhaps we can develop and publish one. I'm all in.

Also, I haven't looked too into it since I have been busy, but this package may do some good:

• grivet

It seems to be active and is based on the same principles this serializer uses, but it's lacking features.

[stefanvanherwijnen]

I haven't worked with TypeScript much but I'll try to implement a deserializer building on your example here. grivet seems nice, but it indeed seems to lack features on the serializer side. I'm looking for a more modular approach (i.e. I want something that simply parses or creates JSON:API documents without any extra features), so ts-japi looks like a nice solution.

Being busy seems to be a global problem so it might also take a while :grin: .

[jrandolf]

I meant to recommend grivet as a deserializer. It only does deserialization. That library combined with this one gives deserialization and serialization :) Both with 0 dependencies, both built on the same principles, so it’s a pretty good pairing.

[stefanvanherwijnen]

Hmm, it mentions sorting, pagination and filtering as missing functionality and that seemed like something purely for the serialize. Afaik these are defined by url params so there is not much to deserializr from the JSON:Api doc.But that supports my modularity standpoint. Parsing an url differs betrween frameworks so it should be performed there instead of the JSON:API package.

[jrandolf]

So the issue I had with a deserializer is the fact that it would be too lame. It should have the functionality of a framework, reason being JSON:API is a framework when it comes to requesting, updating, and creating resources (among other things). In any case, what you say clears my mind a bit. Perhaps a function/class with some lazy loading model resolvers will do well modularly for a deserializer.

[stefanvanherwijnen]

Strictly speaking JSON:API is only a specification. This is why I think a modular JSON:API package should only parse or create documents according to the spec (https://jsonapi.org/format/). The user will then have to determine how to implement e.g. CRUD operations.

I think the deserialzer will simply have to be an interface between the serialized (defined by the spec) and the deserialized (to be defined, e.g. do you place the id along the attributes or in a jsonapi sub-object?) data.

If someone would use an ORM or anything else that would not support the standard deserialized data form, as long as it is strictly specified they could write their own wrapper around it.

[zimmermanw84]

Just to double tap. It would be awesome if the deserializer had no opinions. I.E. Just spits out JSON and lets us implement everything else. IMO this should be just one layer in a framework that could be interchangeable with other serialization strategies should the need arise.

[jrandolf]

JSON:API already spits out a JSON 😂. I think more or less we will take on a graphql-like approach, that is, walk the tree and build resolvers at each Resource and Relationship.