liberation-data / drivine

Best and fastest graph database client (Neo4j, AgensGraph, Apache AGE, FalkorDB) for Node.js & TypeScript.
https://drivine.org
Apache License 2.0
152 stars 33 forks source link

[Question] Possible to initialize with configuration (or any other injected service) and how are environment variables parsed? #94

Open NilsMoller opened 2 years ago

NilsMoller commented 2 years ago

As far as I can tell, there are two options to initialize Drivine:

  1. From environment variables
  2. From values set using a fluent API

From this, I get two questions:

  1. How are environment variables parsed? In the docs, it shows the method DatabaseRegistry.buildOrResolveFromEnv('NEO'). What is 'NEO' in this case? EDIT:
    After digging through the Drivine code, I found that the environment variables are hardcoded and prefixed with the name given (NEO, in this case), resulting in environment variables being forced to take the name NAME_DATABASE_TYPE. Using a configuration package which uses environment variables could conflict with this naming (as it does in my case).
    Specifying how buildOrResolveFromEnv works in the docs might clear things up a bit. I can submit a PR later if you would like me to.

  2. Is there a way to inject configuration (or anything in Nest's DI tree) into the module initialization?
    Nest provides dynamic modules, being able to use DI. Is there a way to use that with Drivine as well?

jasperblues commented 2 years ago

DatabaseRegistry.buildOrResolveFromEnv('NEO'). What is 'NEO' in this case? Does it require a .env file, or can it use environment variables as well?

This is a mechanism for having multiple database connections and assigning a name. There can be:

^-- So NEO here is the name of the database. With the .env style configuration, we declare this by putting a prefix in front of the properties, like eg:

TRAFFIC_DATABASE_TYPE=NEO4J
TRAFFIC_DATABASE_USER='neo4j'
TRAFFIC_DATABASE_PASSWORD='h4ckM3'
TRAFFIC_DATABASE_HOST='localhost'
TRAFFIC_DATABASE_PORT='7687'
TRAFFIC_DATABSE_NAME='neo4j'

^-- This is a database called TRAFFIC with properties suitable for Neo4j.

You can also use the fluent API to define named DBs. You could set it up at runtime. I don't think I provided a sample/example of this, however I helped a client to do it. Pretty easy.

Does it require a .env file, or can it use environment variables as well?

It uses DotEnv (popular config approach) behind the scenes, so whatever works in DotEnv will work. (From memory I think env vars do, right?)

Is there a way to inject configuration (or anything in Nest's DI tree) into the module initialization? Nest provides dynamic modules, being able to use DI. Is there a way to use that with Drivine as well?

You mean to bootstrap Drivine dynamically, or declare connection properties at Runtime? Yeah its possible using the fluent API. Drivine is not that large, so its fairly easy to declare the bits and pieces manually and plug them together. Its also easy to register connections/databases (new ones) on the fly.

I don't think I set it up to support dynamic modules, we easily could though. Which part do you want to configure dynamically?

jasperblues commented 2 years ago

Let me know if I addressed all parts of you question and/or if anything above is not clear.

NilsMoller commented 2 years ago

Thank you for the quick response. I was updating my question as you were typing :)

It uses DotEnv (popular config approach) behind the scenes, so whatever works in DotEnv will work. (From memory I think env vars do, right?)

Indeed it does use env vars. As I mentioned in the updated question, ConnectionProperties#ConnectionPropertiesFromEnv uses process.env already, so both approaches will work just fine.

You mean to bootstrap Drivine dynamically, or declare connection properties at Runtime? Yeah its possible using the fluent API. Drivine is not that large, so its fairly easy to declare the bits and pieces manually and plug them together. Its also easy to register connections/databases (new ones) on the fly.

Yes, to bootstrap it dynamically. For example, Nest's GraphQL library uses a factory to allow for DI to work, simply by injecting anything into the factory method. This can inject, for example, a configuration service. A similar approach could be taken here:

DatabaseRegistry.getInstance()
  .builder()
  .withType(DatabaseType.NEO4J)
  .protocol(config.db.protocol)
  .host(config.db.host)
  .port(config.db.port)
  .userName(config.db.username)
  .password(config.db.password)
  .register(config.db.dbName)

Where config would be injected.
I mention doing it this way, as my configuration package has separators to parse environment variables to config classes, making it incompatible with Drivine's way of using environment variables.
I'm not sure how to approach this. Perhaps you could point me in the right direction?

Bastianowicz commented 2 years ago

I've played around with imports and exports on that as well but couldn't seem to find a working solution. I have tried to create a custom provider on DrivineModule that uses an async factory and injected the configService to that. Unfortunately than the depending modules down the river complain Please make sure that the argument PersistenceManager:NEO at index [0] is available

I am not sure where to do the async stuff here in the module configuration. nestjs-mailer does supply something like this in the config which is pretty straight forward (at least to my eyes)

MailerModule.forRootAsync({
            useFactory: async (configService: ConfigService) => ({
        // dynamic and async config goes here, using the config service
    }, inject: [ConfigService])
})
jasperblues commented 2 years ago

@Bastianowicz What you can do in the meantime is inject the factory instead, like this:

    private persistenceManager: PersistenceManager

    constructor(factory: PersistenceManagerFactory) {
        this.persistenceManager = factory.get(myLateRegisteredDatabase)
    }

I reproduced the issue you folks are having on a current project. It occurs when:

And above is how I worked around that for now.