ClickHouse / ClickHouse

ClickHouse® is a real-time analytics DBMS
https://clickhouse.com
Apache License 2.0
37.8k stars 6.94k forks source link

Data isn`t being inserted with using materialized view #68803

Open RoVys opened 3 months ago

RoVys commented 3 months ago

Hello.

I have an issue with using materialized view. I have a table owoxbisessions. Occasionally I insert data into the table from s3 object storage using a query like this:

    INSERT INTO marketing_raw_data.owoxbi_sessions_
    SELECT *
    FROM s3('https://storage.yandexcloud.net/dataoffice-marketing/test/20240530/*.json.gz',
        'access_key', 'secret_key',
        'JSONEachRow')

A DDL query of owoxbisessions is kinda complicated, I put here only a part of it:

CREATE TABLE marketing_raw_data.owoxbi_sessions_
(
    `user` String,
    `clientId` String,
    `sessionStartTime` String,
    `ptnDate` Date,
    `sessionId` String,
    `device` Tuple(deviceCategory String,
 browser String,
 browserVersion String,
 operatingSystem String),
`customDimensions` Array(Tuple(index Array(Int64), value Array(String)))
)
ENGINE = ReplicatedMergeTree()
PARTITION BY ptnDate
ORDER BY sessionStartTime
SETTINGS index_granularity = 8192;

Full source table DDL: ddl_source_table_for_ticket.txt

My materialized view DDL query(also complicated): mv_for_ticket.txt

I tested the MV query. it works and inserts data into the destination table perfectly.

Destination table DDL: ddl_dst_table_for_ticket.txt

Whats the possible reason of the insert trigger doesnt work?

den-crane commented 3 months ago
CREATE MATERIALIZED VIEW marketing_raw_data.test_mw ON CLUSTER 'cl'
TO marketing_raw_data.owoxbi_sessions
....

FROM marketing_raw_data.test  ---<<<<   test ?
RoVys commented 3 months ago
CREATE MATERIALIZED VIEW marketing_raw_data.test_mw ON CLUSTER 'cl'
TO marketing_raw_data.owoxbi_sessions
....

FROM marketing_raw_data.test  ---<<<<   test ?

Sorry, wrong script. I tried with correct name. It doesnt work. So it`s actually FROM marketing_raw_data.owoxbisessions

den-crane commented 3 months ago

Check that MatView exists at the server where you execute the insert INSERT INTO marketing_raw_data.owoxbi_sessions_.

I suggest to observe the trace of the insert in /var/log/clickhouse-server/clickhouse-server.log