Introduce Result, Record and Graph types mapping

bigmontz commented 1 year ago

⚠️ This a preview feature

Getting Started

Let's say we have the following Cypher query:

 MATCH (p:Person)-[:ACTED_IN]->(m:Movie)<-[:ACTED_IN]-(c:Person) 
 WHERE id(p) <> id(c) 
 RETURN p AS person, m AS movie, COLLECT(c) AS costars

and we are going to load each of the records into Typescript objects like these:

class ActingJobs {
    constructor(
        public readonly person: Person,
        public readonly movie: Movie,
        public readonly costars: Person[]
    ) {
    }
}

class  Movie {
    constructor(
        public readonly title: string,
        public readonly released?: number,
        public readonly tagline?: string
    ){
    }
}

class Person {
    constructor (
        public readonly name: string,
        public readonly born?: number
    ) {

    }
}

Each record in the results will result in an instance of ActingJob with the properties populated from the query results and type validation.

To do this at present, you would write something like the following:

const { records: actingJobsList } = await driver.executeQuery(QUERY, undefined, {
    database: 'neo4j',
    resultTransformer: neo4j.resultTransformers.mappedResultTransformer({ map: (record: Record): ActingJobs | undefined => {
        const person = fromNodeToPerson(record.get('person'))
        const movie = fromNodeToMovie(record.get('movie'))
        const costars = fromNodeListToPersonList(record.get('costars'))

        return new ActingJobs(person, movie, costars)
    } })
})

function fromNodeToPerson (node: any): Person {
    if (isNode(node)) {
        if (typeof node.properties.name !== 'string') {
            throw Error('Person.name is not a string')
        }

        if (node.properties.born != null &&  typeof node.properties.born !== 'bigint') {
            throw Error('Person.born is not a number')
        } 

        return new Person(node.properties.name, Number(node.properties.born))
    }
    throw Error('Person is not a valid node')
}

function fromNodeToMovie (node: any): Movie {
    if (isNode(node)) {
        if (typeof node.properties.title !== 'string') {
            throw Error('Movie.title is not a string')
        }

        if (node.properties.release != null && typeof node.properties.release !== 'bigint') {
            throw Error('Movie.release is not a string')
        }

        if (node.properties.tagline != null && typeof node.properties.tagline !== 'bigint') {
            throw Error('Movie.tagline is not a string')
        }

        return new Movie(node.properties.title, node.properties.release, node.properties.tagline)        
    }
    throw Error('Movie is not a valid node')
}

function fromNodeListToPersonList (list: any): Person[] {
    if (Array.isArray(list)) {
        return list.map(fromNodeToPerson)
    }
    throw Error('Person list not a valid list.') 
}

Using the new mapping functionality, the same result will be achieved using the following code:

const personRules: Rules = {
    name: RulesFactories.asString(),
    born: RulesFactories.asNumber({ acceptBigInt: true, optional: true })
}

const movieRules: Rules = {
    title: RulesFactories.asString(),
    release: RulesFactories.asNumber({ acceptBigInt: true, optional: true }),
    tagline: RulesFactories.asString({ optional: true })
}

const actingJobsRules: Rules = {
    person: RulesFactories.asNode({
        convert: (node: Node) => node.as(Person, personRules)
    }),
    movie: RulesFactories.asNode({
        convert: (node: Node) => node.as(Movie, movieRules)
    }),
    costars: RulesFactories.asList({
        apply: RulesFactories.asNode({
            convert: (node: Node) => node.as(Person, personRules)
        })
    })
}
const { records: actingJobsList } = await driver.executeQuery(QUERY, undefined, {
    database: 'neo4j',
    resultTransformer: neo4j.resultTransformers.hydratedResultTransformer(ActingJobs, actingJobsRules)
})

The mapping will be done by combining rules definition and object properties created by the constructors. It's not necessary to use both (constructor and rules), but the usage of one of them is needed if you need a filled object since properties not present in instantiated object or rules are ignored by the method.

For example, if values present in the Movie and Person objects are never null and no validation is needed. The code to process the result can be changed to:


const actingJobsRules: Rules = {
    person: RulesFactories.asNode({
        convert: (node: Node) => node.as(Person)
    }),
    movie: RulesFactories.asNode({
        convert: (node: Node) => node.as(Movie)
    }),
    costars: RulesFactories.asList({
        apply: RulesFactories.asNode({
            convert: (node: Node) => node.as(Person)
        })
    })
}
const { records: actingJobsList } = await driver.executeQuery(QUERY, undefined, {
    database: 'neo4j',
    resultTransformer: neo4j.resultTransformers.hydratedResultTransformer(ActingJobs, actingJobsRules)

Another possible scenario is Movie, Person and ActingJobs be just typescript interfaces. The code to process the result can be changed to:

const personRules: Rules = {
    name: RulesFactories.asString(),
    born: RulesFactories.asNumber({ acceptBigInt: true, optional: true })
}

const movieRules: Rules = {
    title: RulesFactories.asString(),
    release: RulesFactories.asNumber({ acceptBigInt: true, optional: true }),
    tagline: RulesFactories.asString({ optional: true })
}

const actingJobsRules: Rules = {
    person: RulesFactories.asNode({
        convert: (node: Node) => node.as(personRules)
    }),
    movie: RulesFactories.asNode({
        convert: (node: Node) => node.as(movieRules)
    }),
    costars: RulesFactories.asList({
        apply: RulesFactories.asNode({
            convert: (node: Node) => node.as(personRules)
        })
    })
}
const { records: actingJobsList } = await driver.executeQuery(QUERY, undefined, {
    database: 'neo4j',
    resultTransformer: neo4j.resultTransformers.hydratedResultTransformer<ActingJobs>(actingJobsRules)
})

Note: In this scenario, the ActingJobs interface is set to the transformer for auto-complete and code validation, however this is not treated as an object constructor.

Mapping from `Result` method

Result.as is introduced to the result for enable the mapping occur in the result. Same rules of usage of constructor and rules are share with the hydratedResultTransformer transformer`.

const { records: actingJobsList } = await session.executeWrite(tx => tx.run().as(ActingJobs, actingJobsRules))

Note: The current implementation transforms the Result in a promise. However, this should be update to transform the Result in a MappedResult with the hydrated objects being exposed instead of records. So, it will be possible to async iterate, subscribe and call other methods also.

Direct mapping from an `Record`, `Node` and `Relationship` instance.

Record.as, Node.as and Relationship.as are introduced to enable the mapping of these types only. Same rules of usage of constructor and rules are share with the hydratedResultTransformer transformer`.

const actingJobs = record.as(ActingJobs, actingJobsRules) // example with construtor and rules
const person = node.as<Person>(personRules) // example using interfaces
const actedIn = relationship.as(ActedIn) // example without rules

Note: In Node and Relationship only properties are mapped. _Note: Properties not present in the Record will throw error since Record.get is used under the hood. However, properties which present and with value equals to null or undefined should only throw error when property is not defined as optional in the rules.

Rules and Built-in Rules Factory

The driver provides RuleFactories for all data types which can be returned form a query. There is no need for creating custom rules in most of cases. However, all the built-in rules are extensible.

const personRules: Rules = {
    name: RulesFactories.asString({ 
       convert: name => `Name: $name` // change the convert method in the string rule
    }),
    born: RulesFactories.asNumber({ acceptBigInt: true, optional: true })
}

The Rule interface is defined as:

interface Rule {
  optional?: boolean
  from?: string
  convert?: (recordValue: any, field: string) => any
  validate?: (recordValue: any, field: string) => void
}

where

optional (default: false): indicates with value accept undefined/null as value. In this case, the convert method will not be called.
from (default: property name on the Rules object): indicates from which field on the Record, Node or Relationship the value will came from. Used for cases where the property name in domain object is different to the one present in the database.
convert: called to convert value to the domain value. Called after value be validated. If optional is true and the value is null or undefined, the method is not called.
validate: called to validate if values is valid. Called before convert and after check if the value is optional. If optional is true and the value is null or undefined, the method is not called.

⚠️ This a preview feature

AndyHeap-NeoTech commented 1 year ago

Is there a default hydrator that doesn't require the user to specify the rules?

bigmontz commented 1 year ago

Is there a default hydrator that doesn't require the user to specify the rules?

Yes, but if the user is returning graph types is not that useful, unless they are okay with working with graph types in their domain.

An example of usage is the query:

MATCH (p:Person)
RETURN p.name AS name, p.born AS born

The user can do:

const { records: personList } = await driver.executeQuery(QUERY, undefined, {
    database: 'neo4j',
    resultTransformer: neo4j.resultTransformers.hydratedResultTransformer(Person)
})

However, in this scenario the user will not get validation about types, since the type information can not be extracted in runtime. This information is lost when the code is converted to javascript.

A solution for not having to write the rules it to have a annotation framework and it can be https://github.com/adam-cowley/neode-ogm or some solution using graphql to define the structures and then some function to process and generate the rules for the driver use.

This can be done in light-ogm on top of the driver, because it's maybe a bit too much for the driver get typescript specific annotations.

bigmontz commented 1 year ago

Alternatively, the javascript driver might copy a strategy come from the .net driver. Create a register to associate rules with types, this way you can register once and reuse everywhere. Maybe also selecting differently by Record or Node, since the information can be returned slightly different in some cases.

bigmontz commented 1 year ago

About a global registry for rules, I've got something like:

// Defining the rules
const personRules: Rules = {
    name: RulesFactories.asString(),
    born: RulesFactories.asNumber({ acceptBigInt: true, optional: true })
}

const movieRules: Rules = {
    title: RulesFactories.asString(),
    release: RulesFactories.asNumber({ acceptBigInt: true, optional: true }),
    tagline: RulesFactories.asString({ optional: true })
}

const actingJobsRules: Rules = {
    person: RulesFactories.asNode({
        convert: (node: Node) => node.as(Person)
    }),
    movie: RulesFactories.asNode({
        convert: (node: Node) => node.as(Movie)
    }),
    costars: RulesFactories.asList({
        apply: RulesFactories.asNode({
            convert: (node: Node) => node.as(Person)
        })
    })
}

// registering the rules
neo4j.mapping.register(Person, personRules)
neo4j.mapping.register(Movie, movieRules)
neo4j.mapping.register(ActingJobs, actingJobsRules)

Running the query:

const { records: actingJobsList } = await driver.executeQuery(QUERY, undefined, {
    database: 'neo4j',
    resultTransformer: neo4j.resultTransformers.hydratedResultTransformer(ActingJobs)
})

I didn't implemented any segmentation by driver type (Record, Node, Relationship). However, it's totally possible. The registered rule is used if none is set.

The registry is global to the whole memory, it's not attached to the driver instance.

bigmontz commented 1 year ago

Domain Object Mapping [Javascript]

neo4j / neo4j-javascript-driver