risingwavelabs / risingwave

Best-in-class stream processing, analytics, and management. Perform continuous analytics, or build event-driven applications, real-time ETL pipelines, and feature stores in minutes. Unified streaming and batch. PostgreSQL compatible.
https://go.risingwave.com/slack
Apache License 2.0
7.07k stars 582 forks source link

bug: alter FS source rate_limit cannot apply to MV #19433

Closed tabVersion closed 6 days ago

tabVersion commented 1 week ago
          It works but it breaks in another way. 

Test locally

dev=> CREATE source diamonds (
    carat FLOAT,
    cut TEXT,
    color TEXT,
    depth FLOAT,
) WITH (
    connector = 'posix_fs',
    match_pattern = 'data*.csv',
    posix_fs.root = 'e2e_test/source_inline/fs/data',
    source_rate_limit = 0
) FORMAT PLAIN ENCODE CSV ( without_header = 'false', delimiter = ',');
CREATE_SOURCE
dev=> create materialized view mv as select * from diamonds;
CREATE_MATERIALIZED_VIEW
dev=> select * from mv ;
 carat | cut | color | depth
-------+-----+-------+-------
(0 rows)

dev=> ALTER source  diamonds SET source_rate_limit TO DEFAULT;
ALTER_SOURCE
dev=>
dev=> flush;
FLUSH
dev=>  select * from mv ; <- still not consuming 
 carat | cut | color | depth
-------+-----+-------+-------
(0 rows)

_Originally posted by @tabVersion in https://github.com/risingwavelabs/risingwave/pull/19338#discussion_r1846459451_