apache / paimon

Apache Paimon is a lake format that enables building a Realtime Lakehouse Architecture with Flink and Spark for both streaming and batch operations.
https://paimon.apache.org/
Apache License 2.0
2.45k stars 961 forks source link

[spark] Replace create existing tag semantic with replace_tag #4346

Closed askwang closed 4 weeks ago

askwang commented 1 month ago

Purpose

In default way, the create_tag procedure for an existing tag means to update it or fail, it's semantic is not very accurate, replace it with repalce_tag procedure.

before semantic

CALL paimon.sys.create_tag(table => 'test.T', tag => 'test_tag', snapshot => 1)")

-- success. update it
CALL paimon.sys.create_tag(table => 'test.T', tag => 'test_tag', snapshot => 1, time_retained => '1 d')")

-- failed
CALL paimon.sys.create_tag(table => 'test.T', tag => 'test_tag', snapshot => 2)")

after semantic

CALL paimon.sys.create_tag(table => 'test.T', tag => 'test_tag', snapshot => 1)")

-- failed
CALL paimon.sys.create_tag(table => 'test.T', tag => 'test_tag', snapshot => 1, time_retained => '1 d')")

-- success. use replace to update
CALL paimon.sys.replace_tag(table => 'test.T', tag => 'test_tag', snapshot => 2, time_retained => '1 d')")

Tests

API and Format

Documentation

askwang commented 1 month ago

addressed comments, thanks for review @wwj6591812 A little difference, keep the name 'replace_tag' not change. I think it's name is ok, and iceberg also uses 'replace tag' to update tag. what do you think?

wwj6591812 commented 1 month ago

addressed comments, thanks for review @wwj6591812 A little difference, keep the name 'replace_tag' not change. I think it's name is ok, and iceberg also uses 'replace tag' to update tag. what do you think?

ok