search

cncf / landscape-graph

CNCF Landscape Graph, data model, and applications.
https://github.com/orgs/cncf/projects/7/views/6
Other
39 stars 11 forks source link

Create GraphQL endpoint, leveraging neo4j graphql library v3 #4

Closed halcyondude closed 2 years ago

halcyondude commented 2 years ago

GraphQL schema --> source of truth

Tasks

Moved to new/other issue(s):

More Info

resources

halcyondude commented 2 years ago

after doing a bit of research, a few things are clear.

Game Changer

https://neo4j.com/docs/graphql-manual/current/type-definitions/interfaces/#_directive_inheritance

Any directives present on an interface or its fields will be "inherited" by any object types implementing it. For example, the type definitions above could be refactored to have the @relationship directive on the actors field in the Production interface instead of on each implementing type as it is currently:

interface Production {
    title: String!
    actors: [Actor!]! @relationship(type: "ACTED_IN", direction: IN, properties: "ActedIn")
}

type Movie implements Production {
    title: String!
    actors: [Actor!]!
    runtime: Int!
}

type Series implements Production {
    title: String!
    actors: [Actor!]!
    episodes: Int!
}

interface ActedIn @relationshipProperties {
    role: String!
}

type Actor {
    name: String!
    actedIn: [Production!]! @relationship(type: "ACTED_IN", direction: OUT, properties: "ActedIn")
}

https://neo4j.com/docs/graphql-manual/current/type-definitions/interfaces/#_overriding

In addition to inheritance, directives can be overridden on a per-implementation basis. Say you had an interface defining some Content, with some basic authorization rules:

interface Content
    @auth(rules: [{ operations: [CREATE, UPDATE, DELETE], allow: { author: { username: "$jwt.sub" } } }]) {
    title: String!
    author: [Author!]! @relationship(type: "HAS_CONTENT", direction: IN)
}

type User {
    username: String!
    content: [Content!]! @relationship(type: "HAS_CONTENT", direction: OUT)
}

type PublicContent implements Content {
    title: String!
    author: [Author!]!
}

type PrivateContent implements Content
    @auth(rules: [{ operations: [CREATE, READ, UPDATE, DELETE], allow: { author: { username: "$jwt.sub" } } }]) {
    title: String!
    author: [Author!]!
}
AlexxNica commented 2 years ago

As we get back on track, here are some notes I had about finding the best way of handling the data model<->graphql<->neo4j workflow:

From what I'm seeing right now, the 4-graphql-endpoint-v1 branch isn't handling any types for the GraphQL schema yet. Do you have any thoughts on how to handle that if using GraphQL as the source of truth?

These are the methods I've come so far after some research:

Cypher as the source of truth

This method changes the flow to be Cypher->Neo4j->GraphQL.

Steps:

  1. Use Cypher queries within .cypher files (so we can handle version control, CI/CD) as the actual source of truth.
  2. Load that into the database.
  3. Introspect and generate the GraphQL schema (check if the schema already exists and only generate if changes were made to the types (otherwise we can be generating schemas way too often–possibly extra load).
  4. Generate TypeScript definitions based on the generated schema.
  5. We're ready to consume the GraphQL schema and the TypeScript definitions.

Possible drawbacks:

GraphQL as the source of truth

This method changes the flow to be GraphQL->Cypher->Neo4j. (The Cypher in this case is automatically generated by the Neo4j GraphQL Library.)

Schema:

  1. Write the GraphQL schema (either manually or through apps like arrows.app (see note about arrows.app below))
  2. Use the Neo4j GraphQL Library to integrate the schema through Neo4j and a GraphQL server.
  3. Generate TypeScript definitions based on the schema.
  4. We're ready to consume the GraphQL schema and the TypeScript definitions.

Data loading:

  1. Write mutations directly to the GraphQL server.

Data processing:

Visual representation

It might be worth exploring Mermaid as an option to generate visual representations of the schema. GitHub now supports it natively within markdown instances (issues, documents, etc.).

Note about arrows.app

After some research and testing, I've found arrows.app to be unreliable (some bugs caused the graph to lose its current status and saved outdated versions on top of new ones) and hard to use in a collaborative setting (not having integrations or clear ways of openly collaborating).

More context: https://github.com/neo4j-labs/arrows.app/issues/55#issuecomment-1072335211

Example using Mermaid:

flowchart TB
    subgraph Person
    Person_id["id: ID!"]
    Person_name["name: String!"]
    Person_organization["organization: String!"]
    Person_location["location: [Location!]!"]
    end
    subgraph Role
    Role_id["id: ID!"]
    Role_name["name: String!"]
    end
    subgraph Organization
    Organization_id["id: ID!"]
    Organization_name["name: String!"]
    Organization_headquarters["headquarters: [City!]!"]
    end
    subgraph Location
    Location_id["id: ID!"]
    Location_name["name: String!"]
    end
    subgraph GitRepo
    GitRepo_id["id: ID!"]
    GitRepo_name["name: String!"]
    end
    subgraph TocRole
    TocRole_id["id: ID!"]
    TocRole_name["name: String!"]
    end
    subgraph Project
    Project_id["id: ID!"]
    Project_name["name: String!"]
    end
    subgraph ProjectRole
    ProjectRole_id["id: ID!"]
    ProjectRole_name["name: String!"]
    end
    subgraph TAGrp
    TAGrp_id["id: ID!"]
    TAGrp_name["name: String!"]
    end
    subgraph TagRole
    TagRole_id["id: ID!"]
    TagRole_name["name: String!"]
    end
    subgraph City
    City_id["id: ID!"]
    City_name["name: String!"]
    end
    Person_location -->|LOCATED_IN| Location
    Person -->|MAINTAINER_OF| GitRepo
    Person -->|HAS_TOC_ROLE| TocRole
    Person -->|HAS_ROLE| ProjectRole
    ProjectRole -->|SERVED| Project
    Project -->|IN_SCOPE| TAGrp
    TagRole -->|SERVED| TAGrp
    Organization_headquarters -->|HQ_IN| City
    %% Temporary fix to represent bidirectional linking. It seems GitHub doesn't support Mermaid's native bidirectional linking yet.
    Organization <--> node((IS_SUB)) <--> Organization
    Organization -->|EMPLOYED| Person
    Person -->|IS_BOARD| Organization

Edit: forgot to add the source for the Mermaid example. Here it is: https://github.com/nikas-org/collab-coordination/blob/d795485e6c54c895838a663bbfb55b22cbd9e1b2/examples/landscape-graph_mermaid.md

halcyondude commented 2 years ago

(from CNCF slack #landscape-graph channel) https://cloud-native.slack.com/archives/C03BXBYFMQS/p1658122893053639

Apologies for the long response delay. This is more of a link rosetta stone, I'm working on actual docs that explain some of this, but wanted to elaborate on some of the results of the past 4 weeks of research/learning/design.

Start here

Apollo Federation (v2)

A federated supergraph uses multiple "types" of GraphQL schemas:

(source: https://github.com/apollographql/federation/blob/main/docs/source/federated-types/overview.mdx)

graph TB;
  serviceA[Subgraph<br/>schema<br/>A];
  serviceB[Subgraph<br/>schema<br/>B];
  serviceC[Subgraph<br/>schema<br/>C];
  composition[["🛠<br/>Composition "]];
  supergraph{{"Supergraph schema<br/>(A + B + C + routing machinery)"}};
  api(["API schema<br/>(A + B + C)"]);
  serviceA & serviceB & serviceC --> composition;
  composition -- "(Composition succeeds)" --> supergraph;
  supergraph -- "(Remove routing machinery)" --> api;
  class composition tertiary;

Federation 2 Demo App

https://github.com/apollographql/supergraph-demo-fed2

image

Visualizing and documenting

Visualizing Neo4J Model (Closely Tracks GraphQL…generated from it…)

Current state of neo4j + graphql

CI/CD tooling

Subscriptions

Apollo Federation 2 doesn’t support subscriptions OOTB, have been checking this out as well: https://github.com/apollosolutions/federation-subscription-tools

halcyondude commented 2 years ago

From what I'm seeing right now, the 4-graphql-endpoint-v1 branch isn't handling any types for the GraphQL schema yet. Do you have any thoughts on how to handle that if using GraphQL as the source of truth?

@AlexxNica re: the branch and graphql types, (a couple weeks ago) a prototype is here:

https://github.com/cncf/landscape-graph/blob/4-graphql-endpoint-v1/db/core/landscape-graph-core-schema.gql

Actively working on something more complete this morning.

I've also found this useful for interactive iteration, in concert with Neo4j Desktop's GraphQL extension..

Domain Graph for VSCode

image
halcyondude commented 2 years ago

At the risk of "marketing," pictures > words sometimes :) Hope this helps!

image

image

halcyondude commented 2 years ago

https://github.com/cncf/landscape-graph/blob/4-graphql-endpoint-v1/db/cncf/cncf.graphql

https://github.com/cncf/landscape-graph/tree/4-graphql-endpoint-v1/db/cncf

@AlexxNica FYI (WIP - but progress all the same :))