databricks / iceberg-rest-image

Simple project to expose a catalog over REST using a Java catalog backend
Apache License 2.0
110 stars 46 forks source link

Image doesn't use in-memory database per default #68

Closed Sejmou closed 2 months ago

Sejmou commented 10 months ago

The documentation of this image on DockerHub mentions that it uses an 'in memory backend'. However, I tested it locally and noticed that the catalog was in fact persisted (i.e. it still existed after rebooting the container). I assume this is due to an error in the source code.

I think the connection string jdbc:sqlite:file:/tmp/iceberg_rest_mode=memory in

catalogProperties.putIfAbsent(
        CatalogProperties.URI, "jdbc:sqlite:file:/tmp/iceberg_rest_mode=memory");

needs to be replaced. Currently, this creates an iceberg_rest_mode=memory file in the tmp directory of the container that is persisted and is just a regular sqlite DB file.

manuzhang commented 8 months ago

From the README example, this looks to be on purpose, since you can already list namespaces on start.

dargmuesli commented 2 months ago

This seems to be fixed in the migration to apache/iceberg: https://github.com/apache/iceberg/pull/10908/files#diff-3e0c3a54cb6a1c1992b7aeaf60aaf8df6457f51f2968ff2311b9b0fd827cb150R71

Fokko commented 2 months ago

Kudo's to @danielcweeks 🙌