Propose plugin system for modules

[valiafetisov]

Goal

Modules that are defined in a separate npm package/github submodule

Context

Currently, we expect that all logic modules will need to be defined in the same codebase as the API itself. That might not be optimal if we want to keep it a "boilerplate" – something that advanced users can clone and add their own logic to it.

Therefore, this issue should investigate, document and propose how to separate "modules" into a different repo that is easy to plug based on some config. Notes:

• We suppose to prevent (or maybe at least detect name conflicts)
• The "plugin"/"module" might receive prisma object or ctx object or some new object which will provide obstructed way to database/upcoming "drive" system
  • This actually might mean that the API also needs to become a "plugin". For example: user wants to develop a new "module": they install switchboard-boilerplate as a package in order to test the local module, then add other remote modules

Tasks

• [x] Research and document modularization strategies
• [x] If possible (and straightforward) create PoC

[KirillDogadin-std]

To ensure i get this correctly, pls confirm or deny:

• we are not planning on making the whole package a single module, we would rather make the breakdown in such a way that e.g. has a module exposes some predefined Entities and ways to handle them such as [User, Session, ...]. Or e.g. we would break down https://github.com/powerhouse-inc/switchboard-boilerplate/blob/main/api/src/graphql/context.ts so that there's a more generic way to build the context (i.e. if we need auth then one would import Session module that also has the function/config that would contribute to generating the context object)

Research and document modularization strategies

kinda struggling with this. People just filled google with meta articles about how it should be like but i don't see anything that resembles a real-life example. Would you be fine with me writing a practical proposals that references some of the meta articles that define these general rules?

[valiafetisov]

we are not planning on making the whole package a single module

Yes and no. Ideally, it's both. a) Modules can import switchboard to develop them; b) switchboard can import modules to serve them in production. But to simplify this issue and make it step-by-step, we can go with NO and only focus on b).

Would you be fine with me writing a practical proposals that references some of the meta articles that define these general rules?

I think no articles are really needed to be referenced here, you can just jump into PoC in a PR. How it can look like:

1. Create a new folder in the root called module-example with it's own ts setup (can be same as in api folder or tsup)
2. Implement some module-like logic inside module-example folder
   • Eg resolvers
   • Tables
3. Install module-example into api via npm install ./module-example
4. Import (ideally single) dependency (probably something like setup function) from module-example
   • Call it on switchboard-api start/compilation
   • It should then: provide schema that will be added to the switchboard-api, provide resolvers to be called when schema is used
5. Provide instruction on how to create and install modules

When writing this, it actually feels very similar to the composition, with the exception that authentication and database storage should be part of the switchboard, while the rest (including authorization) should be defined inside module-example.

[KirillDogadin-std]

Eg resolvers Tables

this e.g. is not clear. Can you elaborate or link somethign?

provide schema that will be added to the switchboard-api, provide resolvers to be called when schema is used

can you specify what exactly you mean by the resolver? I.e. is this the contents of the file such as this one or just the functions that process the request such as this one?

authentication and database storage should be part of the switchboard

This is not exactly clear. Should prisma operations be replaceable? I.e. should the module user be able to define their own db operations which will be used in the places where we currently operate with prisma?

I might be blowing the scope here but it generally seems that we either have to rework all the db-related functionality or come up with a solution on how the users will be able to integrate the prisma object and schema we've defined to the existing service. Is it the latter? or something else?

[valiafetisov]

this e.g. is not clear. Can you elaborate or link somethign?

Sorry that it still isn't clear. I think I meant to write graphql resolvers and graphql types (not tables).

I will try to explain it once again from a different perspective: There are modules in our application. Each module is currently located in a separate folder inside ./src/modules/<some_module_name>. After this issue is done, we should be:

• Able to define modules in external npm packages, eg ./src/node_modules/<some_module_name> as well as it is now
• Able to explicitly control what modules are used by the api (eg have config which imports each internal or external module and uses them in the code)

can you specify what exactly you mean by the resolver? I.e. is this the contents of the file such as this one or just the functions that process the request such as this one?

This is not exactly clear. Should prisma operations be replaceable? I.e. should the module user be able to define their own db operations which will be used in the places where we currently operate with prisma?

Those are all good questions, but it's part of the issue is to answer them: as I didn't research it, I honestly don't know what's the minimum implementation would be, due to the complexity of graphql compilers and prisma. For the scope: start with the minimum implementation that makes it possible to define modules outside of the modules folder, clearly specify the shortcomings and how difficult it is to overcome them. Then I can help in terms of scoping (ie should we do it in the same PR, the other or we don't need it at all).