Question: How to solve N+1 problem with Rejoiner

[xiaomeiwen]

I recently face a performance issue in a multi-level query using Rejoiner. Here's an example similar to my data structure:

companyEmployees(startDate, endDate) {
  company {
    id
    department {
      id
      team {
        id
        employee {
          id
        }
      }
    }
  }
}

Assume each level is 1-many. I have defined a set of gRPC APIs to get details at each level, and all the IDs are globally unique (e.g. team 123 only belongs to a specific department, which belongs to a specific company):

getCompanyList()
getDepartmentsByCompanyId()
getTeamsByDepartmentId()
getEmployeesByTeamId()

I then created a query called companyEmployees(companyId) which will use Rejoiner's SchemaModification to handle nested query based on the data structure. It goes like this:

    @Query("companyDepartments")
    ImmutableList<Department> getDepartmentsByCompany(DepartmentByCompanyRequest request,
                                                  DepartmentConfigServiceBlockingStub client) {
        List<Department> departments = client.getDepartmentsByCompany(request).getDepartmentsList();
        return ImmutableList.copyOf(departments);
    }

    @SchemaModification(addField = "teams", onType = Department.class)
    ImmutableList<Team> departmentTeams(DepartmentConfigServiceBlockingStub client, Department department) {
        // query team details by department ID
        request = TeamByDepartmentRequest.newBuilder().setDepartmentId(department.getId()).build();
        teams = client.getTeamsByDepartment(request).getTeamsList();
        return ImmutableList.copyOf(teams);
    }
    @SchemaModification(addField = "employees", onType = Team.class)
    ImmutableList<Employee> teamEmployees(DepartmentConfigServiceBlockingStub client, Team team) {
        // similar logic
    }

I added logs to trace the companyDepartments query and how it triggers SchemaModification. In my DB, I have 1 company, 2 departments, each department contains 2 teams, and each team contains 5 members. And I found out the following behavior: It get the company first -> call SchemaModification to get first department -> call SchemaModification to get first team in first department ->... It looks like a DFS, and the query time becomes linear, based on how much data is there in DB.

Is there any approach I can change my query time in to a constant time, let those queries happen parallel?

Facebook use Dataloader to solve this n+1 prob, I wonder how to solve it with rejoiner. https://engineering.shopify.com/blogs/engineering/solving-the-n-1-problem-for-graphql-through-batching

[LIU0472]

rejoiner dataloader: https://github.com/google/rejoiner/tree/master/examples/src/main/java/com/google/api/graphql/examples/library

[BitPhinix]

Working link: https://github.com/google/rejoiner/blob/master/examples-gradle/src/main/java/com/google/api/graphql/examples/library/graphqlserver/DataLoaderModule.java