StarRocks / starrocks

StarRocks, a Linux Foundation project, is a next-generation sub-second MPP OLAP database for full analytics scenarios, including multi-dimensional analytics, real-time analytics, and ad-hoc queries.
https://starrocks.io
Apache License 2.0
8.29k stars 1.68k forks source link

_statistics_.column_statistics getting created with "replication_num" = "1" #31012

Closed ansh-agrawal closed 3 months ago

ansh-agrawal commented 9 months ago

Steps to reproduce the behavior -

  1. Bring up 6 instances - 3 backend, 3 frontend(where one has envoy proxy set up).
  2. Connect to any of the frontend nodes and add the remaining 2 observers and 3 backends using below respective commands - ALTER SYSTEM ADD FOLLOWER ":"; ALTER SYSTEM ADD BACKEND ":";

First all the observer nodes were added and then the backend nodes.

The frontend and backend nodes starts showing 3 instance each upon show frontends and show backends

Expected behaviour - As the frontend and backend nodes are 3 each now, the _statistics_.column_statistics table should have 3 replicas by default, but that is not happening.

Real behaviour - --statistics`.column_statistics definition

CREATE TABLE column_statistics ( table_id bigint(20) NOT NULL COMMENT "", partition_id bigint(20) NOT NULL COMMENT "", column_name varchar(65530) NOT NULL COMMENT "", .... ) ENGINE=OLAP PRIMARY KEY(table_id, partition_id, column_name) DISTRIBUTED BY HASH(table_id, partition_id, column_name) BUCKETS 10 PROPERTIES ( "replication_num" = "1", "in_memory" = "false", "enable_persistent_index" = "false", "replicated_storage" = "true", "compression" = "LZ4" );`

Not just column_statistics, but all the tables under statistics db show the same behaviour.

The possible issue that we can have is when the backend node contaning this data goes down, causing this table to become corrupt.

StarRocks version - 3.1.0

github-actions[bot] commented 3 months ago

We have marked this issue as stale because it has been inactive for 6 months. If this issue is still relevant, removing the stale label or adding a comment will keep it active. Otherwise, we'll close it in 10 days to keep the issue queue tidy. Thank you for your contribution to StarRocks!