Closed TennyZhuang closed 1 year ago
I don't think we need a logic
keyword to distinguish it from materialized view. The word materialized
is used to distinguish it from logical one.
Rough idea:
Currently, the create sink
statement can only create from a materialized view with the following syntax as an example:
statement ok
create sink si
from
s5 with (
kafka.brokers = '127.0.0.1:29092',
kafka.topic = 'sink_target',
format = 'append_only',
connector = 'kafka'
)
Since there are many stateless ETL cases out there that may only want to sink the output to external systems, whether a materialized view should be created is an option up to users.
Also, some recent benchmarking results show that right now, having MV (even disabling the compactor) may introduce 30-40% overhead than not having MV in terms of throughput.
Is it possible to have improved results later? It is possible, but right now we can just disable MV to show better results than other streaming systems, which is critical in the near future.
In https://github.com/risingwavelabs/risingwave-test/issues/26, it is proposed that we can create sink from view
. Syntax-wise, there are no changes to be made for create sink
if we adopt this approach.
Once View
is supported, we then can execute the query create sink from view
with connector = 'blackhole'
easily.
For blackhole
, I believe we just need to remove the MaterializedExecutor
when generating the executable plan.
Integration with dbt
also requires the support of create view
, right? @neverchanje
As discussed offline (https://risingwave-labs.slack.com/archives/C034U0NH5ND/p1654588390981939), to support the functionality at the beginning, we choose to store the original SQL as a temporary solution. The main concern of storing logical plan is that we don't want to persist it because it may subject to change.
Some notes:
Hi, any progress?
feat(fe,meta): support logical view #6023
Approved in https://github.com/risingwavelabs/risingwave/pull/6023
Is your feature request related to a problem? Please describe.
Currently, RisingWave supports
CREATE MATERIALIZED VIEW
but notCREATE VIEW
. It will be easier to generate some similar tests between logical view and batch view if we support it.Describe the solution you'd like
Support
CREATE VIEW
.Describe alternatives you've considered
As a first-class mview database,
CREATE VIEW
may be ambiguous,CREATE LOGICAL VIEW
is also OK.Additional context