AlexR2D2 / duckdbex

The Library embeds C++ DuckDB database into you Elixir application.
MIT License
72 stars 16 forks source link

Error: "Can't convert DuckDB value of type 'FLOAT[1536]' to the Erlang term." #19

Closed monotykamary closed 4 months ago

monotykamary commented 4 months ago
Error: "Can't convert DuckDB value of type 'FLOAT[1536]' to the Erlang term."

It's definitely likely that the values and terms need to be updated to support array types. I'm trying to work it out on my fork, but I have no idea what I'm doing lol.

AlexR2D2 commented 4 months ago

Could you provide simple code snippet to reproduce the bug?

monotykamary commented 4 months ago

My bad my bad. Here's a simple script to reproduce the bug:

#!/usr/bin/env elixir

Mix.install([
  {:duckdbex, "~> 0.3"}
])

defmodule TestingDuckDBArrays do
  def main() do
    {:ok, db} = Duckdbex.open()
    {:ok, conn} = Duckdbex.connection(db)
    {:ok, _res} = Duckdbex.query(conn, "CREATE TABLE example(embeddings FLOAT[1024])")

    # We insert without a prepared statement as DuckDB doesn't support fixed arrays as parameters
    embeddings = List.duplicate(0, 1024)
    Duckdbex.query(conn, "INSERT INTO example VALUES (#{serialize_array(embeddings)})")

    # Where the bug happens:
    {:ok, res} = Duckdbex.query(conn, "SELECT * FROM example")
    data = Duckdbex.fetch_all(res)
    IO.inspect(data)
  end

  defp serialize_array(array) do
    "[#{Enum.join(array, ", ")}]"
  end
end

TestingDuckDBArrays.main()
{:error, "Can't convert DuckDB value of type 'FLOAT[1024]' to the Erlang term."}
AlexR2D2 commented 4 months ago

Fixed in release v0.3.1

monotykamary commented 4 months ago

Sincerest thank you 🙏 it works perfect! The fix looks like it ended up being much simpler than I thought 🥳