search

datahub-project / datahub

The Metadata Platform for your Data and AI Stack
https://datahubproject.io
Apache License 2.0
9.94k stars 2.94k forks source link

Delta lake ingestion failed (can't parse the string value 0E-18 to decimal) #10418

Open alplatonov opened 6 months ago

alplatonov commented 6 months ago

Describe the bug When attempting to read or interact with a Delta table that contains a decimal datatype column with a value of 0, operations fail. The error is "Exception: Parser error: can't parse the string value 0E-18 to decimal." This issue prevents any interaction with tables that include such decimal values.

To Reproduce Steps to reproduce the behaviour:

  1. Create a table in delta lake format with field decimal(38,18)
  2. try to read and see the error

Expected behaviour The expectation is to be able to read from and interact with Delta tables regardless of the presence of decimal columns with a value of 0.

Screenshots

Additional context pip freeze - pip.txt data created with spark 3.4.2 and delta lake version 2.4.0

github-actions[bot] commented 5 months ago

This issue is stale because it has been open for 30 days with no activity. If you believe this is still an issue on the latest DataHub release please leave a comment with the version that you tested it with. If this is a question/discussion please head to https://slack.datahubproject.io. For feature requests please use https://feature-requests.datahubproject.io

alplatonov commented 5 months ago

datahub version 0.13.1. The bug is still present.

alplatonov commented 4 days ago

The bug is still present.