Nested queries and relation DSL for Read Models

Feature Request

Description

GraphQL allows expressing relations by nesting types like this:

type Employee {
  id: ID!
  name: String!
  company: Company // An employee belongs to a company
}

type Company {
  id: ID!
  name: String!
  employees: [Employee] // A company have many employees
}

It would be a nice addition to support this kind of semantics while defining read models.

Possible Solution

Note about the current proposal: Initially I thought about creating implicit relations between read models that would work similarly to GraphQL's, but that would interfere with the current ability to nest other types and classes. Sometimes, it's useful to aggregate data from several entities in a single read model, and it's useful to be able to nest the types freely. It could also become a real source of headaches for anyone trying to nest a read model type within another one just to find how the framework is injecting an unexpected implicit relation under the hoods. For these reasons I think that it's better to build relationships explicitly with a DSL.

Implement relation-aware decorators in read models to define simple relationships between tables. For example:

`@References(<ReadModelClass>)` constructor parameter decorator

Tags a UUID property as a foreign key from another read model. It adds a relation in the GraphQL schema generated and enables nested queries. If the name of the field matches the name of the referenced read model, the decorator parameter can be omited. The name of the relation generated is the same name of the field minus the "Id" suffix.

@ReadModel
export class Employee {
  public constructor(
    readonly id: UUID,
    readonly name: string, 
    @References readonly companyId
  ) {}

  @Projects(...)
  public static projectionName(...) { ... }
}

`@HasMany({referenceKey: <fieldName>})` property decorator

Tags an array property as a one-to-many relationship with another read model. We assume that the other read model will have a reference targeting ours. If the name of the foreign key in the other table is the same name as our class name plus the "Id" suffix we can omit the referenceKey parameter, as it can be automatically inferred.

@ReadModel
export class ChatGroup {
  @HasMany
  public employees: Array<Employee> = []

  public constructor(
    readonly id: UUID,
    readonly name: string
   ) {}

  ...
}

`@HasOne({referenceKey: <fieldName>)` property decorator

Similar to @HasMany but can be applied to a non-array field to set a reverse reference (It uses the foreign key from a different read model)

@ReadModel
export class Car {
  public constructor(
    readonly id: UUID,
    readonly name: string,
    @References(Driver) readonly driverId: UUID
   ) {}

  ...
}

@ReadModel
export class Driver {
  @HasOne
  public car?: Car 

  public constructor(
    readonly id: UUID,
    readonly name: string
   ) {}

  ...
}

Also, it would be interesting that relations are properly supported by the underlying database by creating indexes to make nested queries performant.

Many to many relationships are harder to implement because they would require generating intermediate tables, and they can be implemented by creating a join read model that keep a reference to the two read models that you want to relate, so I'll leave them out of the scope of this issue.

I love the proposed solution. I think that the interface is easy to understand and simple.

I want to raise some related concerns that I think should be taken into account soon or later:

With some providers, like AWS, there is no limitation when creating Global Secondary Indexes (GSI) when you create the stack for the first time using CloudFormation. However, if you update an already created stack, you can only make one GSI update (creation/deletion) per template update. It would be nice to have a way of doing those updates in the future without that kind of compromise, maybe by doing partial updates.
I think that Booster should also support nested queries pagination, because there are a lot of use cases that would benefit from it when using the @HasMany({referenceKey: <fieldName>}) decorator.

Great DSL Javi. Some things to take into consideration in case they help:

Regarding `@HasMany`

1.- (this is a bit of an implementation detail, but could affect the DSL) I see you followed an "relational database" approach, where the foreign key is stored in the other end of the relationship (the other read model). In non-relational databases, it is the first read model the one which has an array of IDs of the other read models it is related to. If we follow this approach, make each read model store the references to those it is related too, we could find some simplifications in the implementation side (at least for AWS). For example, we won't need an intermediate table for many to many relationships (although this is always this way in non-relational DBs)

2. I see that the type of the field that is decorated with HasMany is of type Array<{other read model}. I think this is correct, as the read model is the representation of what the user will get from the GraphQL API. When implementing this, however, we might need to store an array of those read model IDs. Also we need to take into account how the reducers functions will store new relationships with new read models. I mean, Would they insert in the relationship array the ID of the new related read model or the whole instance?

Regarding `@References`

3. To be coherent with the @HasMany decorator and make the read model reflect the data the user will get from the GraphQL API, I think the type of the field decorated with References should be that of the referenced read model. I mean, it should represent the instance of the read model, not its ID.

If, on the other hand, we decide to follow the approach of storing just IDs in the read models, then I would change the @HasMany decorator to expect a field with an array of IDs and not instances. Whichever you choose, I would choose the same in both decorators.

If we follow the approach of using the read model types and not the IDs, we can always use the decorators without any parameter, as the type of the field (or the field array) is the read model it is related to.

General

4.- I think we can make the @Rererences decorator to also work for "has many" relationship and get ride of the @HasMany decorator. The meaning is the same, and we can know whether it is a has many or has one relationship just by seeing if the field is an array or not. Example:

// This is a one to one relationship
@References
readonly company: Company

// This is a one to many relationship, as it is an array
@References
readonly company: Array<Company>

5.- Finally, I explore going further in simplifying the DSL: What the user really wants to express is "Hey, when I fetch the Employee read model, I want to tell you that I also want to get the company/ies it belongs to" or "When I fetch this Company, I want to get all its employees". So maybe we don't need to call the reverse relationship, the @HasOne, different. We can always use @References. On the implementation side, this works if we follow the approach in point 1. (let each read model store the references to the others). Example:

@ReadModel
export class Car {
    @References readonly drivers: Array<Driver> // Many-to-Many relationship: a car is used by many drivers
    @References readonly motor: Motor // One-to-One relationship: a car has one motor
    @References readonly motor: Array<Wheel> // One-to-Many relationship: a car has many wheels
    //...
}

@ReadModel
export class Driver {
  @References  public cars: Array<Car> // The other end of the many-to-many: a driver has many cars 
}

@ReadModel
export class Motor {
  @References  public car: Car // The other end of the One-to-One: a motor is in one car
}

@ReadModel
export class Wheel {
  @References  public car: Car // The other end of the One-to-Many, which is equaly expressed as the above: a wheel belongs to one car.
}

That's it, I hope this helps you in some way. Also, Adrian made two great questions that could require some thouths

Agree with @AdrianLorenzoDev 's concerns, but in my opinion that would go for a future iteration.

Also, I agree with @alvaroloes 's simplification by using the @References field. I believe this simplifies stuff given our current implementation, and it is much more clear for the user I believe.

If it helps, you could take a look how References in MongoDB work

Hey team, great questions and thoughts! When I wrote this proposal I was thinking about the immediate use case I need to solve, but I see that relations could open a whole new line of work.

My most immediate problem goes like this: Let's say that I'm creating a "social network" where people can create posts that are related to the creator. I need to put the reference id to the user in the post object because I can't predict the number of posts that a user can create. They could potentially create ∞ posts, well, maybe not ∞, but certainly, enough to end up going over the max object size in my database and producing failures.

Then I thought, well, if I have a relation between my posts and the user that created them, it would also be nice to have the inverse relation and, given a specific user, get the list of their posts. Still, this approach hits the problem that @AdrianLorenzoDev describes: If I have a potentially infinite number of posts, I'd need to paginate that part of the query somehow, and that starts making things a bit too complicated for my taste. Paginating a list from nested queries could trigger many weird edge cases, which makes me think that building this inverse relationship for potentially infinite lists makes no sense. If the posts are properly indexed by creatorID, we can use the regular paginated list query for Posts filtering by creatorID.

Then I read what @alvaroloes wrote about modeling one-to-one, one-to-many or many-to-many relations using high-level types and taking advantage of the NoSQL pattern of storing arrays of IDs, and it definitely makes sense too for use cases where the number of related items is limited, so maybe we should differentiate between these two different use cases:

1. Constructor parameter decorator to make queries by reference keys performant

This one works as the @References decorator described in my original post. It just tags a UUID field as a reference to another table, inserting the index in the DB, but it doesn't make changes in the GraphQL schema; it will just make filtering by the reference identifier faster. This allows us to get a paginated list of posts for a specific user (or a set of users) with good performance without making changes to the current API. Basically, it only creates an index in the tagged key so we could call this decorator @index. Generalizing this to allow indexing any field gives some extra flexibility to end users to decide how they can query their data.

2. Property decorator for read-model typed objects

This one tags a ReadModel-typed field to build NoSQL-style relationships as described by @alvaroloes. Internally, it will store only the IDs of the read model objects, generating the GraphQL relations and not requiring inserting indexes (Because this implementation only looks up tables by primary keys).

The @index decorator could be enough to solve my use case, but we can keep the discussion open about the other kind of relations and how to model them. I think that @alvaroloes's suggestion of using the same decorator for all kinds of relations and using the type to differentiate makes a lot of sense.

In addition to that, it could also make sense to build wrapper classes to make working with relations easier. For instance, developers could work with child instances in a relation object, and Booster would deal with the ID storage and the lower-level details. Something like this:

@ReadModel
export class Driver {
  @References public cars: RelationList<Car> 

  @Projects(SomeEntity)
  public static projectSomeEntity(...) {
    ...
    this.car.push(aCarInstance) // You can append a Car instance, but only the ID is stored
  }
}

Similar wrapper classes can be created for other kinds of relationships too.

boostercloud / booster