elixir-explorer / adbc

Apache Arrow ADBC bindings for Elixir
https://arrow.apache.org/adbc/
Apache License 2.0
50 stars 16 forks source link

Can the driver support custom path? #24

Closed watsy0007 closed 1 year ago

watsy0007 commented 1 year ago

In my workflow, I use python and duckdb quite often, I have been considering how to migrate to Elixir for a while, Recently duckdb officially support the ADBC API and provided examples.

DuckDB ADBC - Zero-Copy data transfer via Arrow Database Connectivity ADBC API

The offical C++ example is as follow:

AdbcDatabaseNew(&adbc_database, &adbc_error);
AdbcDatabaseSetOption(&adbc_database, "driver", "path/to/libduckdb.dylib", &adbc_error);
AdbcDatabaseSetOption(&adbc_database, "entrypoint", "duckdb_adbc_init", &adbc_error);
// By default, we start an in-memory database, but you can optionally define a path to store it on disk.
AdbcDatabaseSetOption(&adbc_database, "path", "test.db", &adbc_error);
AdbcDatabaseInit(&adbc_database, &adbc_error);

Currently, the project only supports a fixed drivers sqlite postgresql flightsql snowflake, Is it possible to add a driver for custom path?

for example:

{:ok, db} =
  Kino.start_child(
    {Adbc.Database, driver: "/path/to/libduckdb-osx-universal/libduckdb.dylib", entrypoint: "duckdb_adbc_init"}
  )
{:ok, conn} = Kino.start_child({Adbc.Connection, database: db})
{:ok, result} = Adbc.Connection.query(conn, "select 1;")

or

{:ok, db} =
  Kino.start_child(
    {Adbc.Database, driver: :custom, path: "/path/to/libduckdb-osx-universal/libduckdb.dylib", entrypoint: "duckdb_adbc_init"}
  )
{:ok, conn} = Kino.start_child({Adbc.Connection, database: db})
{:ok, result} = Adbc.Connection.query(conn, "select 1;")
josevalim commented 1 year ago

Yes, pull requests are welcome!

josevalim commented 1 year ago

Actually, I may have a fix soon!

dlindenkreuz commented 10 months ago

@watsy0007 Were you able to successfully set up DuckDB via ADBC in the end?

watsy0007 commented 10 months ago

@dlindenkreuz yes. as follows

image

references

  1. DuckDB ADBC - Zero-Copy data transfer via Arrow Database Connectivity
  2. ADBC API # C++
  3. DuckDB Installation / Environment C++ / Package Binary
dlindenkreuz commented 10 months ago

Nice, thanks for the swift response. I found out that further keyword options are passed through to the native initializer, so I was able to specify the DB path like this:

defmodule MyApp.Application do
  def start(_type, _args) do
    children = [
      {Adbc.Database,
       driver: "./libduckdb.dylib", # relative to cwd
       entrypoint: "duckdb_adbc_init",
       path: "./.data/duck.db",
       process_options: [name: MyApp.DuckDB]},
      {Adbc.Connection, database: MyApp.DuckDB, process_options: [name: MyApp.DuckConn]},
      # ...