search

AlexR2D2 / metabase_duckdb_driver

Metabase DuckDB Driver shipped as 3rd party plugin
Apache License 2.0
75 stars 25 forks source link

Error loading shared library with DuckDB driver #3

Closed seajhawk closed 2 years ago

seajhawk commented 2 years ago

Originally opened as https://github.com/metabase/metabase/issues/25789, but redirected here.

Describe the bug Opening issue per @flamber in this thread which has more details: https://discourse.metabase.com/t/duckdb-support/20092/2

I'm trying to use the DuckDB driver but running into issues. I'm running in a linux docker container. I've tried with a single duckdb file, and I've tried with no file specified.

Both return this error in the UI: image

Logs [b44c25d9-af89-4e2d-b840-64218b2c0d0c] 2022-09-28T16:32:33-07:00 DEBUG metabase.server.middleware.log POST /api/database 400 1.6 s (0 DB calls) {:message "/tmp/libduckdb_java16810484802722090474.so: Error loading shared library ld-linux-x86-64.so.2: No such file or directory (needed by /tmp/libduckdb_java16810484802722090474.so)"}

To Reproduce Steps to reproduce the behavior:

  1. Install the DuckDB driver with Metabase in a docker container per https://github.com//AlexR2D2/metabase_duckdb_driver
  2. Connect to a new DuckDB database (can be just a folder path)
  3. See error

Expected behavior I expect Metabase to connect to the DuckDB database and allow me to query it.

Information about your Metabase Installation:

Severity This is a complete blocker on moving my Metabase-based solution from incubation to the next phase. I'm building a free solution to be used by non-tech folks as cheaply and easily as possible. I need the simplicity and performance of DuckDB and its columnar data storage so I can provide both Metabase and the database in a single performant container.

I have a number of folks who are already using the solution, but currently it's Postgres-based and it's too expensive for them to run hosted Postgres and they are not technical enough to install and manage their own Postgres server.

Additional context Add any other context about the problem here.

AlexR2D2 commented 2 years ago

The problem in the Alpine image of Metabase. The DuckDB embeds the icu lib. ICU requeres the glibc pthread lib (not the musl pthread lib installed in the Alpine by default). I tried to install different versions of the glibc library to the Metabase Alpine image but there is some issues with anyway.

Until 2.32-r0 version of the glibc, there is the std::system_error in the icu std::call_once call. I this this is the same error as in https://github.com/microsoft/ClearScript/issues/331.

Starting from 2.33-r0 version of glibc there is "Error relocating /lib/ld-linux-x86-64.so.2: unsupported relocation type 37" error.

So, instead of digging into all of this issues i just created a Ubuntu based image of Metabase and added DuckDB plugin. Please see Docker section of README.md

seajhawk commented 2 years ago

I am closing as I'm following Alex's instructions to build my own docker image.

I believe any further fix needs to be in the Metabase image, not the driver.