ClickHouse / clickhouse-js

Official JS client for ClickHouse DB
https://clickhouse.com
Apache License 2.0
213 stars 26 forks source link

How to parameterize TO clause when creating materialized view? #262

Closed DatGuyJonathan closed 5 months ago

DatGuyJonathan commented 5 months ago

I'm using client@1.0.1 & server@24.1.3. I'm trying to create a table, then a materialized view. The commands & their params look like this, and this works fine:

{
    createTargetTableSql: `
        CREATE TABLE IF NOT EXISTS {targetTableName: Identifier}
        (
            "domain_name" String,
            "month" Date,
            "sumCountViews" AggregateFunction(sum, Int64)
        )
        ENGINE = AggregatingMergeTree
        ORDER BY (domain_name, month)
    `,
    createMaterializedViewSql: `
        CREATE MATERIALIZED VIEW IF NOT EXISTS {materializedViewName: Identifier}
        TO MonthlyAggregatedData_0_0
        AS SELECT
            toDate(toStartOfMonth(event_time)) AS month,
            domain_name,
            sumState(count_views) AS sumCountViews
        FROM {baseTableName: Identifier}
        GROUP BY
            domain_name,
            month
    `,
    query_params: {
        baseTableName: "HourlyData_0_0",
        targetTableName: "MonthlyAggregatedData_0_0",
        materializedViewName: "MonthlyAggregatedDataMv_0_0"
    }
}

If I change TO MonthlyAggregatedData_0_0 into TO {targetTableName: Identifier}, I get this error:

Both table name and UUID are empty. 

If I change TO MonthlyAggregatedData_0_0 into TO {targetTableName: String}, I get this error:

Syntax error: failed at position 110 ('}') (line 2, col 36): }
        AS SELECT
            toDate(toStartOfMonth(event_time)) AS month,
            domain_name,
            sumState(count_views) AS sumCountViews
       . Expected substitution type (identifier).

Is there a specific way to parameterize the TO clause when creating a materialized view?

Please let me know if I can provide more info. Thanks in advance.

slvrtrn commented 5 months ago

I think the minimal repro is this (the client sends the requests similar to the following curls):

curl "http://localhost:8123?param_target=mv_target" \
--data-binary "CREATE OR REPLACE TABLE {target: Identifier} (i Int32) ENGINE MergeTree ORDER BY i;"

(adding an extra src table as it was missing from the OP)

curl "http://localhost:8123?param_src=mv_src" \
--data-binary "CREATE OR REPLACE TABLE {src: Identifier} (i Int32) ENGINE MergeTree ORDER BY i;"

Creating the MV:

curl "http://localhost:8123?param_target=mv_target&param_src=mv_src&param_mv_name=mv" \
--data-binary "CREATE MATERIALIZED VIEW {mv_name: Identifier} TO {target: Identifier} AS SELECT * FROM {src: Identifier};"

Which fails with the same error:

Code: 60. DB::Exception: Both table name and UUID are empty. (UNKNOWN_TABLE) (version 24.3.1.2672 (official build))

I will clarify with the team if it's possible to parametrize the TO ... clause as of now.

slvrtrn commented 5 months ago

It is confirmed that, unfortunately, this is a current limitation of ClickHouse.

slvrtrn commented 5 months ago

@DatGuyJonathan, here's the issue in the main repo to keep track: https://github.com/ClickHouse/ClickHouse/issues/62892

DatGuyJonathan commented 5 months ago

Thank you @slvrtrn!