risingwavelabs / risingwave

Best-in-class stream processing, analytics, and management. Perform continuous analytics, or build event-driven applications, real-time ETL pipelines, and feature stores in minutes. Unified streaming and batch. PostgreSQL compatible.
https://go.risingwave.com/slack
Apache License 2.0
6.8k stars 563 forks source link

bug: should use `executor_id` to count mview creation progress #6236

Open BugenZhao opened 1 year ago

BugenZhao commented 1 year ago

a fragment contains more than one chain operator and we need to take care

Yes. I believe there will some problems with the current MV on MV creation reporting since we're using the actor id. 😢

Originally posted by @BugenZhao in https://github.com/risingwavelabs/risingwave/issues/6233#issuecomment-1305547119

BugenZhao commented 1 year ago

https://github.com/risingwavelabs/risingwave/issues/6270 will be a prerequisite.

github-actions[bot] commented 1 year ago

This issue has been open for 60 days with no activity. Could you please update the status? Feel free to continue discussion or close as not planned.

BugenZhao commented 1 week ago

This is not a problem anymore, as there's no chance that two Backfill executors reside in the same actor. Otherwise,

xxchan commented 1 week ago

In theory, we can have multiple singleton executors in the same fragment (e.g., count(*) mv, or StreamValues). In our current code, it seems this won't happen, since JOIN/UNION will force a hash shard to inputs, and doesn't have the specialization for 2 singleton inputs.