tdikland
commented
6 months ago What I would love to see in this project:
- A Delta Sharing server that could be configured and launched from the command line that is at least at feature parity with the reference server implementation here
- A library with implementation of the Delta Sharing Protocol (i.e. the REST API) that can be completely customised.
The Delta Sharing server implementation is already making good progress and is on its way to be at parity with the reference implementation. In doing so the server did make some assumptions, for example it uses a postgres database to store the shared tables, it includes some admin routes, etc. I think this is great, because it allows people to get up and running quick and easy.
The second point, a library only focused on implementing the protocol, would be aimed at more advanced users who want to customize their Delta Sharing server (e.g. auth middleware, rate limiting, storing shared tables in a different database, etc.). These users could depend on this crate for the core implementation (and maybe also some middleware) and customise it by implementing traits. I have some idea's/findings from experimentation in https://github.com/tdikland/delta-sharing-server-rs.
Regarding organisation, I think it makes sense to have a crate that implements the Delta Sharing protocol, a crate with useful extensions/middleware for the sharing server and finally a reference implementation that depends on the aforementioned crates and that is plug&play.
There have been several discussions about the organisation and scope of this project, for example https://github.com/delta-incubator/delta-sharing-rs/pull/19 and https://github.com/delta-incubator/delta-sharing-rs/pull/22. Let's discuss some options here.