Closed ibabeo closed 2 months ago
Delta-rs version: 0.16.4
Binding: python
Environment:
What happened: Merge on predicate throw error when the merge key is a date column, which table is partitioned by.
How to reproduce it:
import pandas as pd from datetime import date from deltalake import DeltaTable, write_deltalake import tempfile f = tempfile.TemporaryDirectory().name df = pd.DataFrame(data={"Date": [date(2023,1,1)] *10 + [date(2024,1,1)]*10, "B":range(20), "C":"constant"}) write_deltalake(f, df, mode='append', partition_by=["Date"]) dt = DeltaTable(f) df_new = pd.DataFrame(data={"Date":[date(2023,1,1)], "B":[1], "C":"new value"}) dt.merge(source=df_new, predicate='t.Date = s.Date AND t.B = s.B', target_alias='t', source_alias='s').when_matched_update_all().execute()
DeltaError Traceback (most recent call last) ... \lib\site-packages\deltalake\table.py] in execute(self) 1776 Dict: metrics 1777 """ -> 1778 metrics = self.table._table.merge_execute( 1779 source=self.source, 1780 predicate=self.predicate, DeltaError: Generic DeltaTable error: Unable to convert expression to string
DeltaError Traceback (most recent call last) ... \lib\site-packages\deltalake\table.py] in execute(self) 1776 Dict: metrics 1777 """ -> 1778 metrics = self.table._table.merge_execute( 1779 source=self.source, 1780 predicate=self.predicate,
DeltaError: Generic DeltaTable error: Unable to convert expression to string
More details: The same code works in deltalake 0.16.1
This is already fixed here https://github.com/delta-io/delta-rs/pull/2357
And will be available in the next release
Environment
Delta-rs version: 0.16.4
Binding: python
Environment:
Bug
What happened: Merge on predicate throw error when the merge key is a date column, which table is partitioned by.
How to reproduce it:
More details: The same code works in deltalake 0.16.1