Connecting to graph databases

[koalabi]

Description / Steps to reproduce / Feature proposal

I'd like to be able to use lb4 with a graph database. Currently my preferred target would be OrientDB but, even if it were Neo4j (which is probably more mainstream today), I have a hard time finding out where to start (after much googling and reading the loopback website). It would probably be more straightforward with lb3 but it is for a few greencase applications, so I'd very much prefer to use lb4.

My main question is: how could one use a graph database, knowing that 1-to-many relations are not the dominant case (rather many-to-many) and that the model is not relational.

Any hint from the loobpack core team will be appreciated.

Kind regards, Alain

Current Behavior

Expected Behavior

See Reporting Issues for more tips on writing good issues

[bajtos]

Hi @koalabi, thank you for raising this feature request. AFAICT, there is a LB3 connector for Neo4j maintained by our community, see https://www.npmjs.com/package/loopback-connector-neo4j-graph.

Could you please help us to better understand what scenarios/use cases would you like to solve using a graph database and how would you like to encode them in a domain model (model relations)?

[koalabi]

Hi Miroslav,

Thanks for your question and my apologies for the late answer (too busy these days! :-( ).

I mainly target 2 types of applications: one is more of the modeling kind (enterprise architecture models, business process models, software dependencies, ITIL CMDB's), where relations are many and of very diverse nature, and the other one is more related to knowledge modeling/management/linked data ("à la RDF" ;-)).

Moreover, I have needs in the short term and in the medium term. In the short term, it's enough to be able to retrieve the immediate neighborhood of a given node (full or partial list of edges connected to the node with the nodes at the other end). In other words, I don't need nor plan to perform sophisticated queries on the database (such as "shortest path from - to"). Such queries will probably come later on (in the medium term) but they are today only a remote possibility.

I have had a quick look at the lb3 neo4j connector. (It's not OrientDB of course but Neo4j is fine too). It might fit the bill but can I use it "out of the box" with lb4 ?

Kind regards, Alain

[koalabi]

Additional comment, maybe: I don't need the relations to be strongly typed: a string rendition indicating the nature of each edge would be a good start. I imagine the highly dynamic nature of edge types in graph databases would make a type-safer approach a maintenance nightmare and an almost impossible task.

[bajtos]

I have had a quick look at the lb3 neo4j connector. (It's not OrientDB of course but Neo4j is fine too). It might fit the bill but can I use it "out of the box" with lb4 ?

AFAICT from reading the docs, the connector provides regular CRUD methods and allows models to call execute API to traverse relations.

I think that should work pretty well with LoopBack 4. Here is what you can try:

• Define a new datasource via lb4 datasource, enter loopback-connector-neo4j-graph as the custom connector.
• Define a new model via lb4 model
• Define a new repository via lb4 repository
• Create a new CRUD controller

To traverse relations, you can extend the generated repository class with custom methods calling this.modelClass.dataSource.connector.execute.

Unfortunately, DefaultCrudRepository does not implement execute method yet, otherwise you could call execute directly. Would you like to contribute this improvement?

Here is the code to change:

https://github.com/vvdwivedi/loopback-next/blob/24e06ec776560b1adcdb4ffb31d72509d2ad25f6/packages/repository/src/repositories/legacy-juggler-bridge.ts#L336-L344

The code snippet below shows a simplified implementation. The real implementation would have to handle the cases where dataSourceor connector or execute are not defined.

    return new Promise((resolve, reject) => {
      const connector: legacy.Connector = this.modelClass.dataSource!
        .connector!;
      connector.execute!(
        command,
        parameters,
        options,
        (err: any, ...results: any[]) => {
          if (err) return reject(err);
          if (results.length < 1) resolve();
          else if (results.length === 1) resolve(results[0]);
          else resolve(results);
        },
      );
    });

Alternatively, and I think it's a better solution, we can push most of this code into loopback-datasource-juggler, either DataSource or DataAccessObject.

Example usage:

class MyRepository extends DefaultCrudRepository<MyModel, string> {
  async getUsersByEmail(email: string) {
    const results = await this.execute({
        query: 'MATCH (u:User {email: {email}}) RETURN u',
        params: {
          email: 'alice@example.com',
        },
    });

    const result = results[0];
    if (!result) {
        console.log('No user found.');
    } else {
        var user = result['u'];
        console.log(JSON.stringify(user, null, 4));
    }
  }
}

[bajtos]

I opened a pull request in juggler to implement dataSource.execute, see https://github.com/strongloop/loopback-datasource-juggler/pull/1671

[bajtos]

A related question on StackOverflow: How to execute arbitrary SQL query

[franverona]

By mixin @bajtos previous answer and some code of my own, I've created a way to use raw queries on repositories.

Repository Helper

import { DataSource } from "loopback-datasource-juggler";

export default class RepositoryHelper {

  public readonly dataSource: DataSource;

  constructor(dataSource: DataSource) {
    this.dataSource = dataSource;
  }

  async query(sql: string, params?: any, options?: any): Promise<void> {
    return new Promise((resolve, reject) => {
      const connector = this.dataSource.connector!;
      connector.execute!(sql, params, options, (err: any, ...results: any) => {
        if (err) {
          return reject(err);
        }

        if (results.length === 0) {
          return resolve();
        }

        if (results.length === 1) {
          return resolve(results[0]);
        }

        resolve(results);
      });
    });
  }
}

Some repository

import { DefaultCrudRepository, repository } from '@loopback/repository';
import { SomeModel } from '../models';
import { DatabaseDataSource } from '../datasources';
import { inject, Getter } from '@loopback/core';
import RepositoryHelper from './helper.repository';

export class SomeRepository extends DefaultCrudRepository<
  SomeModel,
  typeof SomeModel.prototype.id
  > {

  helper: RepositoryHelper;

  constructor(
    @inject('datasources.db') dataSource: DatabaseDataSource
  ) {
    super(SomeModel, dataSource);

    this.helper = new RepositoryHelper(this.dataSource);
  }

  async query(sql: string, params?: any, options?: any): Promise<void> {
    return await this.helper.query(sql, params, options);
  }
}

Then you can use your repository as follows:

const query = await this.someRepository.query('SELECT * FROM users');
console.log(query)

For example, inside an API call:

import { Filter, repository, Where } from '@loopback/repository';
import { param, get, getFilterSchemaFor } from '@loopback/rest';
import { User } from '../models';
import { UserRepository } from '../repositories';

export class UserController {
  constructor(
    @repository(UserRepository)
    public userRepository: UserRepository,
  ) { }

  @get('/users/{id}', {
    responses: {
      '200': {
        description: 'Get user',
        content: {
          'application/json': {
            schema: { type: 'array', items: { 'x-ts-type': User } },
          },
        },
      },
    },
  })
  async find(
    @param.path.string('id') id: string,
    @param.query.object('filter', getFilterSchemaFor(User)) filter?: Filter,
  ): Promise<any> {
    return await this.userRepository.query(`SELECT * FROM users WHERE id = ${id}`);
  }

}

[prth-123]

@franverona I implemented the above code in my project. It was working fine and also i am able to run raw query. But i can run my application only if i have commented auto migrate line written in index.ts else it is giving error like this :- Can you provide any solution on this?

[franverona]

Sorry @prth-123 but I'm no longer on the "Loopback" ecosystem. This was an small project I worked on long time ago.

[prth-123]

Ok @franverona, Thanks for your support.

[stale[bot]]

This issue has been marked stale because it has not seen activity within six months. If you believe this to be in error, please contact one of the code owners, listed in the CODEOWNERS file at the top-level of this repository. This issue will be closed within 30 days of being stale.

[stale[bot]]

This issue has been closed due to continued inactivity. Thank you for your understanding. If you believe this to be in error, please contact one of the code owners, listed in the CODEOWNERS file at the top-level of this repository.

[albjeremias]

why are all the issues closed automatically in loopback4 next ?!