Contributing a database connector for rqlite: Yay or nay?

joostdecock commented 3 months ago

What would you like to be added

A database connector for rqlite (I am happy to contribute it myself).

Description

I would like to use a HA step-ca deployment with rqlite as database.

My reasons are the same as everyone's: This is the database I'm using for other aspects of the project, so it would be great to also use it for Step-CA.

I have been reading up on what it would take, and saw this comment where @mmalone writes:

if you don’t like MySQL for HA, the DB backend is pluggable and only needs key-value support (no SQL necessary). So one should be able to write a backend for a more HA-oriented database easily :)

Sounds great. I would like to do that, and write a connector for rqlite. I would obviously prefer to donate this connector for inclusion in the product. But I'm a bit concerned by #52 being open for 6 months with @hslatman commenting:

We're currently working on a new version of nosql that changes some things in terms of the exposed interface. It should be available fairly soon. We aim for no / minimal breaking changes, but at this time I can't say for sure what it'll look like. I don't expect big changes to happen to the core of your implementation. It's probably good to await its release and adapt this PR, so that it starts of in a nice way.

Questions

Are you open to a contribution that would implement support for rqlite as a database?
What is the current status of this new version of nosql as it seems to be a blocking factor for new contributions.

Why this is needed

Needed is not the word I'd use. But it's obviously nice to have an additional backend. Especially one which is so easy to setup as rqlite (which is distributed sqlite).

hslatman commented 2 months ago

Hey @joostdecock, thank you for opening the issue.

We've discussed it in our triage meeting, and we concluded that we're open to adding support for Rqlite to this repository, and thus the CA.

The concern you have is a valid one: it's been quite a while since the MongoDB PR was opened, and there hasn't been progress for months now. The rewrite of nosql can be found here: https://github.com/smallstep/nosql/pull/53. I'd say that's feature complete, but can use some user testing, and would need some changes in the CA to implement it. Since the last commits to that branch, we had to shift gears in other areas, so that's why it has been left in this state for this long. We still intend to get it merged, as it is a more robust implementation, so that will happen at some point, but I don't have a concrete timeline at this time.

You could wait till it gets merged, or you can start implementing Rqlite support based on that branch and see if the new API fits well with Rqlite too. Having someone actively working with the new interfaces will likely also help us moving the needle forward.

joostdecock commented 2 months ago

Thank you for your detailed feedback. That's wonderful news that you're open to adding Rqlite support, that really made my day 😊

I''ll have a look at #53 and see what I can learn.

While I can't drop everything and work on this immediately, I am looking forward to submitting a PR and working together to make this happen 🙏

otoolep commented 2 months ago

rqlite creator here, happy to answer any questions about it if it helps in adding support.

smallstep / nosql