search

elixir-explorer / adbc

Apache Arrow ADBC bindings for Elixir
https://arrow.apache.org/adbc/
Apache License 2.0
50 stars 16 forks source link

support for decimals dtype in adbc (duckdb tested) #77

Closed matreyes closed 5 months ago

matreyes commented 5 months ago

Hi there!, I'm having an issue because duckdb defaults to decimal instead float:

:ok = Adbc.download_driver!(:duckdb)
{:ok, db} = Kino.start_child({Adbc.Database, driver: :duckdb})
{:ok, conn} = Kino.start_child({Adbc.Connection, database: db})
Adbc.Connection.query(conn, "SELECT 11.1111;")

>  {:error, %ArgumentError{message: "not yet implemented for format: `d:6,4`"}}

Sadly I couldn't replicate in sqlite, it seems it always uses float (even casting to decimal or numeric).

Thanks!

cocoa-xu commented 5 months ago

Hi @matreyes, thanks for reporting this issue. I’ll try to implement this today. :)

josevalim commented 5 months ago

Perhaps it is worth bringing the Decimal library as a dependency for this one?

cocoa-xu commented 5 months ago

Perhaps it is worth bringing the Decimal library as a dependency for this one?

I agree. It'd be much easier to start from scratch or casting it to other types

matreyes commented 5 months ago

Perhaps it is worth bringing the Decimal library as a dependency for this one?

Wouldn't generate a compatibility issue with explorer?

(I was trying with Explorer.Series.from_list([Decimal.new(1, 42, -1)]) and it fails)

josevalim commented 5 months ago

Explorer does not support a decimal yet but we want to. There is an open issue.

jc00ke commented 5 months ago

Looks like this was recently^1 fixed^2 in adbc

image

Hopefully we can get an adbc 0.3.2 soon! :clap: :clap:

cocoa-xu commented 5 months ago

Hi @jc00ke, my plan is to implement all list types plus the struct one in Adbc.Column in this week (and more types if I have enough time), and then perhaps we can release a new version.

But I'm not sure if @josevalim has any plan for the next release, like if we're only going to release one once all types are supported in Adbc.Column. Below is the list of types that haven't implemented yet: