redpanda-data / redpanda

Redpanda is a streaming data platform for developers. Kafka API compatible. 10x faster. No ZooKeeper. No JVM!
https://redpanda.com
9.65k stars 589 forks source link

iceberg: initial transactions + update_schema_action #23496

Closed andrwng closed 1 month ago

andrwng commented 1 month ago
Introduces the concept of a transaction, which is a collection of
updates to a table to be sent to the catalog together. Updates and their
corresponding requirements are generated via a new `action` abstraction,
which are constructed and run in the context of a single transaction.

The actions update in-memory table_metadata owned by the transaction,
but the actual update to the table is only committed when the
transaction's updates are sent to a catalog (not included in this
commit).

To demonstrate, one such action is included in this commit, for schema
updates.

Backports Required

Release Notes