risingwavelabs / risingwave

Best-in-class stream processing, analytics, and management. Perform continuous analytics, or build event-driven applications, real-time ETL pipelines, and feature stores in minutes. Unified streaming and batch. PostgreSQL compatible.
https://go.risingwave.com/slack
Apache License 2.0
7.06k stars 581 forks source link

refactor: revisit column ID assignment #17340

Open xxchan opened 5 months ago

xxchan commented 5 months ago

During work in https://github.com/risingwavelabs/risingwave/pull/17293, we can see it's a mess. It's error-prone and we cannot understand how it works precisely.

A little more background:

xxchan commented 2 months ago

Another factor to consider: In iceberg it's required to have a field_id, including nested types' fields (should be unique in the table schema), which will be used in schema evolution and column projection.