search

vendure-ecommerce / vendure

The commerce platform with customization in its DNA.
https://www.vendure.io
Other
5.63k stars 995 forks source link

RFC: Fork TypeORM #3102

Open michaelbromley opened 14 hours ago

michaelbromley commented 14 hours ago

Background

TypeORM is the library we use in Vendure to interact with the database. It is a highly performant (comparing very favourably to Prisma and Drizzle in these benchmarks published by Prisma), is feature-rich, and mature. It currently has around 1.9 million weekly downloads on npm.

However, the library is currently effectively unmaintained. At this time of writing, it has had no commits for 7 months and no releases since January (9 months ago).

There are 2.4k open issues and 147 open PRs. Several of those PRs are fixing critical issues that are causing bugs in Vendure:

In addition to that, there are now critical vulnerabilities being flagged up by code scanning tools are flagging up a high-impact vulnerability in the current version:

All of this creates a number of problems for Vendure:

Options

Although TypeORM works well for us (the above bugs notwithstanding), the current situation is not sustainable. Therefore something must change. Here are the options we have considered:

Migrate to an alterative ORM

There are a number of other ORM solutions such as MikroORM, Prisma & Drizzle. In themselves they are all worthy options and if we were starting out from scratch they would all be viable options. However, considering our large userbase - many of which are enterprise companies with lots of custom code in the form of Vendure plugins, there are a couple of huge downsides to this option:

  1. It will be a massive job to change out TypeORM from core. 673 uses of the this.connection, not to mention other TypeORM-specific features we use such as subscribers and custom decorators.
  2. The same will be true for user code (on a smaller scale). But virtually every Vendure plugin will make use of the TypeORM APIs in some way.
  3. Moving to a new library comes with its own set of unknowns. It is one thing to evaluate the APIs and do a proof-of-concept, but only after extended, real-world use do the true characteristics of performance, DX etc become apparent. TypeORM has the huge advantage here in that it is very well known and understood.

This option has a significant risk of creating a "python 2/3"-style split in the community, as existing, large-scale Vendure applications cannot justify the significant time investment in migrating, the plugin ecosystem becomes split, and existing knowledge and practices become obsolete. For this reason I am ruling out the option of migrating to a different ORM.

Wait for the TypeORM community to fix things

In a very long thread on The Future of TypeORM (where these issues are discussed at length), the situation of lack of maintenance has been a topic since way back in 2019.

More recently, a community member announced that they are working with the creator of TypeORM to address this situation by putting together a community-driven team of maintainers.

We had high hopes for this but unfortunately, since the announcement in July 2024, there has been no further communication on this effort. I have also personally reached out to the volunteer (who is also a member of the Vendure community) but cannot get any details of what, if anything, has been done on this.

Four months with no visible forward progress is too long and we are no longer confident that this effort will successfully resolve the TypeORM maintenance problems.

Fork it

This leaves us with the option of forking TypeORM and maintaining our own version. This is also not a decision to be taken lightly, which is why I am writing this RFC because it affects all Vendure users.

A fork would have the following goals:

  1. Address urgent issues by getting fixes in to the library that are already languishing in unattended PRs
  2. Resolve high-impact vulnerabilities
  3. Simplify the project by removing core support for MongoDB, SAP HANA and potentially MS SQL & Oracle (to be evaluated)
  4. Begin to clean up the code base to make it easier to maintain
  5. Keep a regular release cadence, with fixes merged and released in a timely manner.

Non-goals:

  1. Adding support for any other DB types
  2. Changing the API
  3. Becoming the de-facto successor to TypeORM for all TypeORM users. I can imagine that some will want to switch if we do this right. But we will be very clear about the strict scope of the fork as primarily exiting to serve Vendure's needs. This is necessary for us to be able to maintain it sustainably.

Impact

Forking TypeORM would have the following impact on Vendure users (non-exhaustive list - feedback welcome):

  1. In all your plugin code that uses TypeORM APIs would need to update to the new package name in imports. I checked a real-world large project and we have 27 imports in total across 30 plugins. So this is probably not a large task for most.
  2. In monorepos that use TypeORM outside of the Vendure context, you will have to decide whether to also migrate other projects to our TypeORM fork and update their imports accordingly.

Pros

Cons

Your Feedback

This is a decision that will affect all Vendure users. Therefore we want to hear from you. What are you thoughts on this topic?

After a period of gathering feedback, assuming that a fork seems to be the best option, we will then plan in detail how it will work and how that will tie in with the Vendure roadmap.

martijnvdbrug commented 14 hours ago

Not great, not terrible...

Ideally, we would use a mature library, maintained by 'the world' (I've heard good stories about Sequalize). But, no matter what library we choose, the amount of effort required to migrate will be HUGE, for Vendure and for all consumers with customizations.

Maintaining a vendure-typeorm fork is easier now, but the entire maintenance burden will land on Vendure Core team + community. I have no idea how much time is going into maintenance for such a fork, is there sense of this?

Can I simplify it like this?

tianyingchun commented 11 hours ago

why not migrated to prisma?

taxilian commented 6 hours ago

On the pro column for maintaining your own fork, if the TypeORM community does get a real successor you could still switch back to that and not lose much time and work. On the other hand, if you migrate to a totally different library and then TypeORM picks up again you'd have to switch things back to use it.

Rough decision. Personally I'd rather use MongoDB, but I know I'll be in the extreme minority there ;-)

taxilian commented 4 hours ago

Thinking about this a bit more, there is another question to ask:

What is the cost of switching? What is the cost of changing your decision later?

What is the cost of doing your own fork? What is the cost of changing your decision later?

It seems like doing your own fork has a very small cost right now and changing from that to a different framework in 6 months or a year if that becomes worthwhile isn't really going to be much more "expensive" than if you switch now. In other words, if the best long term choice isn't clear now then it may make sense to go with the cheapest option since it won't be more "expensive" later to fix vs now. Technical debt is a beast, but by staying with what we have you don't increase it by a huge amount, at least for the project itself. I suppose the same can't be said for the people using it who may need to update their code to work with the new vendure later.

I think my vote would be to stick with your own fork and keep it to just bugfixes and/or required features while you wait to see if typeorm is going to pick up again. If it doesn't then you can still make the change later on.

arrrrny commented 3 hours ago

I used mikro-orm with Nestjs before. It is very similar to type-orm. Back in the time when I was using it was faster than typeorm. Reason for my db heavy project. I think this can also be a good opportunity to decouple pieces in the core system and use only repository and services, and implement db operations as high level details. I think core being this much tightly coupled to a DB system makes it rigid and fragile. I would like to help if you decide a greenfield SOLID project.