elixir-explorer / adbc

Apache Arrow ADBC bindings for Elixir
https://arrow.apache.org/adbc/
Apache License 2.0
50 stars 16 forks source link

Materializing decimal column fails when null values present #102

Closed mckuzyk closed 1 month ago

mckuzyk commented 1 month ago

If a %Adbc.Column{} has decimal dtype and null values are present, materializing the column will fail due to pattern matching a bitstring against a nil value.

column = %{
data: [#Reference<0.2680975598.2998796290.32494>, #Reference<0.2680975598.2998796290.32505>],
name: "NULLABLE_COL",
type: {:decimal, 128, 38, 0},
metadata: %{
"byteLength" => "4",
"charLength" => "0",
"finalType" => "T",
"logicalType" => "FIXED",
"precision" => "38",
"scale" => "0"
},
__struct__: Adbc.Column,
nullable: true
}

Adbc.Column.materialize(column)

Leads to a match error

** (MatchError) no match of right hand side value: nil
    (adbc 0.6.1) lib/adbc_column.ex:1202: anonymous fn/3 in Adbc.Column.handle_decimal/3
    (elixir 1.17.2) lib/enum.ex:1703: Enum."-map/2-lists^map/1-1-"/2
    (adbc 0.6.1) lib/adbc_column.ex:1193: Adbc.Column.handle_decimal/1

Adding a pattern to pass nil values through fixes the problem in my local tests:

 defp handle_decimal(decimal_data, bits, scale) do
    Enum.map(decimal_data, fn data ->
      case data do
        <<decimal::signed-integer-size(bits)-little>> ->
          if decimal < 0 do
            Decimal.new(-1, -decimal, -scale)
          else
            Decimal.new(1, decimal, -scale)
          end

        nil -> nil
      end
    end)
  end
cocoa-xu commented 1 month ago

Hi @mckuzyk, thanks for reporting this issue! It should be addressed in https://github.com/elixir-explorer/adbc/pull/103, I'll release a new version once CI passed!

cocoa-xu commented 1 month ago

v0.6.2 is shipped!

mckuzyk commented 1 month ago

Just tested and it works perfectly. Thanks again!