delta-io / delta-rs

A native Rust library for Delta Lake, with bindings into Python
https://delta-io.github.io/delta-rs/
Apache License 2.0
2.22k stars 394 forks source link

Decimal Column with Value 0 Causes Failure in Python Binding #2193

Closed bnewman-tech closed 3 months ago

bnewman-tech commented 7 months ago

Environment

Delta-rs version: 0.15.3

Binding: Python

Environment:


Bug

What happened:

When attempting to read or interact with a Delta table that contains a decimal datatype column with a value of 0, operations fail. The error is "Parser error: can't parse the string value 0E-10 to decimal." This issue prevents any interaction with tables that include such decimal values.

What you expected to happen: The expectation is to be able to read from and interact with Delta tables regardless of the presence of decimal columns with a value of 0.

How to reproduce it:

from decimal import Decimal
from deltalake import write_deltalake
import pandas as pd

# Create a DataFrame with a decimal column
df = pd.DataFrame({"num": [1, 2, 3], "letter": ["a", "b", "c"]})
df["decimal_col"] = [Decimal(0), Decimal("1.0000000000"), Decimal("2.0000000000")]

# Display the DataFrame
print(df)

# Attempt to write the DataFrame to a Delta table
write_deltalake(
    "libs/deltalake/tables/some-table", df, overwrite_schema=True, mode="overwrite"
)

Error Message:

Exception has occurred: Exception Parser error: can't parse the string value 0E-10 to decimal

leo-schick commented 5 months ago

It looks to me that this is the same issue as #2221