Entity collection querying

[danReynolds]

Experimental support for accessing and filtering entity collections by typename from the Apollo cache using the new useFragment and useFragmentWhere hooks.

This change introduces the concept of data entity collections, grouped by type. It is currently difficult to perform queries for entities of a particular type from the Apollo cache, such as all Employee entities in the cache with age == 42. Since the cache does not group entities by type, you would need to create a pseudo collection field by writing a field to the root query and always keeping it up to date when new entities of that type are added/removed.

If you then wanted to perform the filter by age, you would need to write another type policy called employeesAboveAge that filters the pseudo collection down further.

This ends up being a lot of overhead and developer maintenance to do manually and could be made easier. This feature supports automatic collection grouping of entities in the cache, which can then be queried using the new useFragment and useFragmentWhere as well as the watchFragment and watchFragmentWhere APIs.

an example of using the hooks might look like this:

  const { data } = useFragment(
    gql`
      fragment GetEmployee on Employee {
        id
      }
    `,
    {
      id: 'Employee:1',
    }
  );

  const { data } = useFragmentWhere(
    gql`
      fragment GetEmployeesByAge on Employee {
        id
        first_name
      }
   `,
    {
      employee_age: 42,
    }
  );

This is still an experimental implementation requiring some further cleanup/testing but the PR is up early for feedback from folks.

Proposed implementation

Whenever a normalized entity like an Employee:1 object is written to the cache, a reference is also added to a CacheExtensionsCollectionEntity entity for that type, with ID CacheExtensionsCollectionEntity:TypeName that is shaped like this:

{
  "ROOT_QUERY": {...},
  "CacheExtensionsCollectionEntity:Employee": {
    data: [{ __ref: 'Employee:1']}
    __typename: "CacheExtensionsCollectionEntity",
    id: "Employee"
  },
  "Employee:1": {...},
}

With the list of entities by type now maintained in the cache, we can then add support watching for updates to the full or filtered collection. While there is a matching readFragment API available as a counterpart to readQuery, there is no built-in useFragment or watchFragment API for useQuery or watchQuery. This is likely because entities themselves are not watchable in the Apollo cache, only queries support the observable API necessary for subscribing to changes.

To overcome this limitation and introduce the useFragment, watchFragment and new useFragmentWhere APIs, we automatically generate a type policy for these calls and access the data from the collection entities. This way, whenever any fields requested in the fragment change in the cache, the type policy will update and deliver new data, just like a manually defined type policy would.

Open API questions

Right now it creates the automatically generated type policies by fragment name. Fragment and query names should be unique across apps since conflicting ones would break tools like TS type generation. The alternative to relying on fragment names here would be to have the new APIs like useFragmentWhere support a name or alias field to cache it under, which can be done but adds a bit more developer overhead.

I will probably list a few more things here as people point them out/I think through why this is all a bad idea 🙃

[sjyoung12]

This looks really creative/promising, @danReynolds !

It is currently difficult to perform queries for entities of a particular type from the Apollo cache, such as all Employee entities in the cache with age == 42. Since the cache does not group entities by type, you would need to create a pseudo collection field by writing a field to the root query and always keeping it up to date when new entities of that type are added/removed. If you then wanted to perform the filter by age, you would need to write another type policy called employeesAboveAge that filters the pseudo collection down further.

Just to make sure I'm correctly oriented within the problem space, is it fair to say that the current proposal solves the same problem as the root query / field policy pattern, but with less boilerplate code required of the developer?

If so, it may be instructive to provide a reference example showing how the motivating example of querying Employees of a certain age would be accomplished under the existing paradigm.

As an aside: this would be a great topic for a future GraphQL WG meeting. 😉

[danReynolds]

This looks really creative/promising, @danReynolds !

It is currently difficult to perform queries for entities of a particular type from the Apollo cache, such as all Employee entities in the cache with age == 42. Since the cache does not group entities by type, you would need to create a pseudo collection field by writing a field to the root query and always keeping it up to date when new entities of that type are added/removed. If you then wanted to perform the filter by age, you would need to write another type policy called employeesAboveAge that filters the pseudo collection down further.

Just to make sure I'm correctly oriented within the problem space, is it fair to say that the current proposal solves the same problem as the root query / field policy pattern, but with less boilerplate code required of the developer?

If so, it may be instructive to provide a reference example showing how the motivating example of querying Employees of a certain age would be accomplished under the existing paradigm.

As an aside: this would be a great topic for a future GraphQL WG meeting. 😉

Yea! we discussed it at a working group meeting a couple weeks back, I think you were out. I'm planning to write up a post demonstrating the use cases it solves, it was getting to be too much background for a PR description.