JanKaul
commented
1 year ago I'll try to implement a catalog that is compatible to the JDBC catalog of the java version. Then you just have to start up a postgres/MySQL Container and it should work.
It might just be a couple of days until I will get around to implementing it.
Hi,
First of all thank you so much for carrying the torch for bringing Iceberg support to Rust!
I've been looking for a simple to deploy Iceberg project that I can just run locally using Docker. Dremio-oss comes close but I'm having an issue with it and in any case, I'm still confused about the Iceberg Catalog.
Long story short, I'm wondering whether we can wrap up what you've done so far in a Docker image, expose some of the functionality in a CLI and run a small webserver to expose the REST API. I'm no Rust expert but I might be able to handle the CLI and Docker parts as I've done that before (https://github.com/prql/prql-query/).
Even if the functionality is currently not complete, my sense is that having something that's easy to deploy and try out might get more people to play with it who might then be inspired to implement missing functionality that they would like to see.
My personal requirements are quite limited as I have folders of parquet files in Minio buckets that I would just like to add to an Iceberg manifest and expose as an Iceberg table. My data arrives in whole files at a time so I have no need for updates or insert so I got the impression that the current state of the project might already be able to handle that use case.
WDYT? I'm eager to help with the Docker part, my biggest limitation is just time. If you could just provide some guidance on the traits and functions that you've implemented then I'll see what I can do.