search

powerhouse-inc / switchboard

Open-source API over the document model made to be developer- and analyst-friendly
https://switchboard-boilerplate.vercel.app
GNU Affero General Public License v3.0
3 stars 3 forks source link

Research existing solutions #2

Closed valiafetisov closed 1 year ago

valiafetisov commented 1 year ago

Goal

Overview on the existing tools that we can use

Context

During the initial planning, we raised the questions and mentioned few existing solutions which might cover some of the requirements. Those include sanity, strapi, slicknode for headless CMSs and postgraphile and graphile for automatically generated graphql endpoints.

Tasks

valiafetisov commented 1 year ago

Headless CMSs

Automatically generated graphql endpoints from SQL database

Bare tools

BracketJohn commented 1 year ago

But

"But", in rejection of what?

Other remarks:

Conclude the very initial technical stack

what I'm missing here at the moment to come to a conclusion is a set of requirements that we have that these tools will help us with. I know we have it in docs and issues elsewhere - can you link them / do a high-level overview of the requirements here or in a dedicated issue as well, so that we can reach a more grounded decision?

valiafetisov commented 1 year ago

Good points, thank you!

what I'm missing here at the moment to come to a conclusion is a set of requirements

You're right, those are the initial requirements I've just put together based on the existing input. Feel free to comment there as well!

But please note, the initial purpose of the current issue was to investigate, if we can use use any existing headless ORMs. And I think the conclusion is clear: we better not if we want to keep flexibility for the long-term features. I agree about spending more time on other "bare tools" and will more thoroughly look into typeorm competitors like prisma, knex.js, nexus, etc

valiafetisov commented 1 year ago

@BracketJohn I've added 5 more libraries to the Bare tools section above and further specified requirements in #3. I think my conclusion stands the same, although MikroORM looks interesting (but also less popular/less integrated). Let me know if you want to see other tools in the list

BracketJohn commented 1 year ago

Generates schema directly from types 😮

wow, this is actually crazy - I think the limitation here then is, that they will not be able to support many features that exist in SQL but not in TS? I see that they for exampke support UUID, but as you said: it's quite complex already. If I imagine something like check-constraints this would probably be a lot more complex.

Thanks for expanding the section. There's no more tooling I'm missing -> which one do you want to go with, based on the requirements collected in #3?

other than that: I think it's time to get going, thjanks for already opening the issues 🎊 Let's do full-steam ahead starting on Monday (:

BracketJohn commented 1 year ago

It requires a complex build-step that will be hard to impossible to modularize

Prisma update: Actually, they have client-side extensions now: This is quite nice as it adds typed alterations to the prisma client. This also allows to implement computed properties: https://www.prisma.io/docs/concepts/components/prisma-client/client-extensions

valiafetisov commented 1 year ago

As discussed in the daily: in order to settle with ORM, we need to finally compare two closest competitors: Prisma vs TypeORM. Here are my requirements

Prisma vs TypeORM

  1. Flexibility. My main fear is that because Prisma models have to be pre-compiled, we might end up with essential feature that they simply don't support yet, or a feature the support of which they will drop one day? Don't have a good example for this yet (maybe enums?)

    • TypeORM: flexible as javascript
    • Prisma: limited to the available language features

  2. Support for extensible model structure. I think that the main quality of the ORM (at least for this particular project) is to not only provide the type-safe interface of the database, but the extensible model to build business logic. In other words, have a central place where we define tables AND actions that can be taken with them.

    • With TypeORM it will look like this:

      // CoreUnit.ts
@Entity()
export class CoreUnit {
    @PrimaryGeneratedColumn()
    id: number

    @Column()
    name: string

    @Column()
    descriptionSentence: string

    async updateDescription(context: Context, descriptionSentence: string): Promise<CoreUnit> {
        await context.roles.verifyIsAdmin()
        await this.save({ descriptionSentence })
        return this
    }
}
// usage
CoreUnit.findOneBy({ id: '123' }).updateDescription()

    • Prisma instead suggests to Wrap a Prisma Model in a Class (but client extensions feature might also help here, they just didn't add it to the same guide):

      // schema.prisma
model CoreUnit {
    id                              String @id @default(cuid())
    name                            String
    descriptionSentence             String
}
      // CoreUnit.ts
class CoreUnit {
    constructor(private readonly CoreUnit: PrismaClient['CoreUnit']) {}

    async updateDescription(context: Context, id: string, descriptionSentence: string): Promise<CoreUnit> {
        await context.roles.verifyIsAdmin()
        return this.CoreUnit.update(id, { descriptionSentence })
    }
}
// usage
const prisma = new PrismaClient()
const coreUnit = new CoreUnit(prisma.CoreUnit)
await coreUnit.updateDescription(context, '123', 'test test')

  3. The ability to dynamically define model. I expect that the end goal of this product (development-wise) is to make a declarative-like interface where in some file we just define how this particular model behaves on a business-level and then everything else is geneated: the endpoints, the interfaces, input validators, etc. For this to work, I expect that we will need to extend or come up with our own declarative structure that will require necessary fields and "dynamically" generated models based on that.

    • TypeORM: there is full flexibility of javascript
    • Prisma: not supported

  4. Split logic info different module folders

  5. Hooks. I expect that some part of the logic in this project will be hooks: do something, when something else changes. The first known feature requiring hooks is record history. The requirement to preserve previous values along with the method name that changed them as well as the user

    • TypeORM: 2 different kinds of hooks: Listeners and Subscribers – basically js- or postgres-based. With different, clearly named decorators that triggered only on certain events
    • Prisma: js-based only, very primitive middleware concept where we need to add boilerplate code to check executed method

  6. Migrations

    • TypeORM: you can write migrations in typescript
    • Prisma: sql-based migration is the norm

  7. Virtual columns

  8. Dynamic data source. In the end, due to the offline-first requirements, we might want to use possibility to dynamically change/set/create duplicate of the datasource based on the locally available data

    • TypeORM: It's not a problem, since it's just javascript, so can be reinitialised at any point, with any data source
    • Prisma: It's actually not a big problem for Prisma as well :tada:
BracketJohn commented 1 year ago

Flexibility. My main fear is that because Prisma models have to be pre-compiled, we might end up with essential feature that they simply don't support yet, or a feature the support of which they will drop one day? Don't have a good example for this yet (maybe enums?)

agreed - with an open end project like this it is probably best to keep all options open

Prisma instead suggests to Wrap a Prisma Model in a Class (but client extensions feature might also help here, they just didn't add it to the same guide):

I think here Prisma is actually on the same level as typeORM with the recent extensions feature. If you look at the docs of this you'll see that you can modify behavior on most levels. I think this should be neough.

The ability to dynamically define model. I expect that the end goal of this product (development-wise) is to make a declarative-like interface where in some file we just define how this particular model behaves on a business-level and then everything else is geneated: the endpoints, the interfaces, input validators, etc. For this to work, I expect that we will need to extend or come up with our own declarative structure that will require necessary fields and "dynamically" generated models based on that.

where is this requirement from?

Split logic info different module folders

as outlined orally: I have not seen this actually happening in https://github.com/makerdao-ses/ecosystem-api instead I've seen https://github.com/makerdao-ses/ecosystem-api/blob/main/seeds/01-CoreUnits.js as a central definition and then in https://github.com/makerdao-ses/ecosystem-api/tree/main/src/modules/Auth it's only about business logic -> where do you get this requirement from?

Virtual columns

as above: this should be resolved by client-extensions (you also write this, I just want to make clear that in my eyes the two solutions are on the same level here)

doing a summary here: I see that in point (1), (3) (if actual requirement), (4) (if actual requirement) typeorm leads. If 3 and 4 are both not requirements it would boil down to (1).

what about considerations like developer-experience and improved typescript support, both of which are probably important to external contributors?

valiafetisov commented 1 year ago

I think here Prisma is actually on the same level as typeORM with the recent extensions feature

Agree with this, extensions feature is quite nice. Although the interface is not as friendly as a wide variety of decorators

The ability to dynamically define model.

where is this requirement from?

There is a requirement to build document interface for every kind of document and every kind of action on top of this document. From this I derive that we would rather make a declarative structure where the interface is generated from it, instead of forcing to implement the UI/validators separately separately for each model and force external developers to do it every time they submit a change.

Split logic info different module folders

where do you get this requirement from?

This was raised during the meeting with Wouter. As far as I understood, ecosystem-api should not be eyed as a quality standard, if we can do better. I personally would prefer to keep those things in one place, but also understand that as soon as there relations, you can no longer clearly separate those things

what about considerations like developer-experience and improved typescript support, both of which are probably important to external contributors?

Overall, I see no major differences here, some things just done differently here vs there. There is also no much difference features-wise (4 is really no a very big deal), but the only real considerations for me here were 1 and 3.

ridafkih commented 1 year ago

Not sure what this is about as I'm just here to the reference on an issue on the Prisma repository, but if y'all need dynamically generated Prisma schémas, I have a library on my profile which would let you generate them programatically. 😂

https://github.com/ridafkih/schemix