search

Electron100 / butane

An ORM for Rust with a focus on simplicity and on writing Rust, not SQL
Apache License 2.0
81 stars 11 forks source link

NoSQL backend #64

Open jayvdb opened 1 year ago

jayvdb commented 1 year ago

I suspect butane will get a lot more interest if it also supported at least one non-SQL based backend. This is something that is a fairly unique opportunity to butane that other Rust ORM's are not suited for.

The easiest would likely be redis or similar caching system, which very likely wouldnt be able to support the butane query system beyond pkey fetches. But it would demonstrate that NoSQL is possible, and people interested in more featureful NoSQL like mongo, etc would have a base implementation to work from.

LoveIsGrief commented 1 year ago

Does any ORM do this? I thought NoSQL meant document store and manual handling of relations (as in the client has to handle everything). That would seem like a massive undertaking to redo all of the relational modeling and handling (especially constraints) in a library written by 1-3 contributors.

Maybe I'm wrong as I don't use any document stores or NoSQL, so feel free to point me to NoSQL databases that would be good candidates that don't require writing DB logic in the client lib (butane). Always happy to learn more.

jayvdb commented 1 year ago

Hi @LoveIsGrief , thanks for raising valid concerns.

My team is already pushing butane models into redis, so this task would involve moving our custom logic into butane, which isnt a small task because butane internals are necessarily more complex, and would need more generic tests created, but it is doable. And we only need to do a basic NoSQL support - I am betting that the butane team will grow when people can see butane can do NoSQL, as they will want to improve/extend it.

Newer ORMs do often provide NoSQL support. For example, Mongo ORMs. Notably on that list, Prisma is Rust underneath and is quite nice, but we selected the less mature butane because it was pure Rust and thus has a similar user-base and set of objectives as we have. https://www.prisma.io/docs/concepts/overview/prisma-in-your-stack/is-prisma-an-orm is a nice explanation of how it, and other ORMs like TypeORM, are not strictly tied to a relational structure.

https://github.com/franklin-ai/butane/commit/6e34de770e9801a816795ad554eb354dfbb9b9c7 is the rather over-simplified approach to avoid re-doing all of the butane internals to allow easy NoSQL support. It uses https://github.com/jayvdb/once_cell_serde instead of the standard once_cell. This approach was taken primarily so that we have few modifications to butane, as we want to stay close to main. I am about to rebase that today, and it will be much smaller now that all of the fake changes are also in main. That commit allows serialising the relations and the related data inline - that means there are extra structures in the JSON that describe the fkeys and many, which are not needed because the related data is embedded. There are many ways to improve it, but it was suffice for our immediate needs.

(update: https://github.com/jayvdb/butane/commit/99056f54633958d9ac1a236011f2b72d91466aa0 is the rebased commit - much cleaner)

jayvdb commented 1 year ago

It would be desirable to do this after https://github.com/Electron100/butane/pull/22 , otherwise it will make that feature harder to complete.