update shared data compose

[DanRoscigno]

• MINIO_ACCESS_KEY is deprecated
• Update to version 3.1 (use CN instead of BE)
• remove persistent volumes as this should be for experiments, not prod use

[DanRoscigno]

Use:

Check the cluster

Check the Compute Node (CN)

Connect to the FE with an SQL client

mysql -P9030 -h127.0.0.1 -uroot --prompt="StarRocks > "

StarRocks > SHOW COMPUTE NODES\G
*************************** 1. row ***************************
        ComputeNodeId: 10004
                   IP: 10.5.0.3
        HeartbeatPort: 9050
               BePort: 9060
             HttpPort: 8040
             BrpcPort: 8060
        LastStartTime: 2023-11-29 01:05:36
        LastHeartbeat: 2023-11-29 01:08:47
                Alive: true
 SystemDecommissioned: false
ClusterDecommissioned: false
               ErrMsg:
              Version: 3.1.5-5d8438a
             CpuCores: 6
    NumRunningQueries: 0
           MemUsedPct: 0.08 %
           CpuUsedPct: 0.1 %
       HasStoragePath: true
          StarletPort: 8167
             WorkerId: 1001
1 row in set (0.01 sec)

Create a MinIO bucket

open http://localhost:9001/buckets

• Click on Create Bucket
• Name the bucket; in the example configuration shown lower on this page the bucket name is starrocks

Create a MinIO keypair

open http://localhost:9001/access-keys

• Click on Create access key
• Click on Create :::note Don't just view the key and secret, it is not created until you click on Create. :::
• After clicking on Create, click on Download for import

DDL for a shared-data storage volume

Details for the configuration shown below:

• The MinIO server is available at the URL http://minio:9000
• The bucket created above is named starrocks
• Data written to this volume will be stored in a folder named shared within the bucket starrocks :::tip The folder shared will be created the first time data is written to the volume :::
• The MinIO server is not using SSL
• The MinIO key and secret are entered as aws.s3.access_key and aws.s3.secret_key, use the ones you created in the MinIO web UI and downloaded
• The volume shared is the default volume

CREATE STORAGE VOLUME shared
TYPE = S3
LOCATIONS = ("s3://starrocks/shared/")
PROPERTIES
(
    "enabled" = "true",
    "aws.s3.endpoint" = "http://minio:9000",
    "aws.s3.use_aws_sdk_default_behavior" = "false",
    "aws.s3.enable_ssl" = "false",
    "aws.s3.use_instance_profile" = "false",
    "aws.s3.access_key" = "npAamJH7naisPNnUMzBW",
    "aws.s3.secret_key" = "E33cdRM9MfWpP2FiRpc056Zclg6CntXWa3WPBNMy"
);

SET shared AS DEFAULT STORAGE VOLUME;

StarRocks > DESC STORAGE VOLUME shared\G

*************************** 1. row ***************************
     Name: shared
     Type: S3
IsDefault: true
 Location: s3://starrocks/shared/
   Params: {"aws.s3.access_key":"******","aws.s3.secret_key":"******","aws.s3.endpoint":"http://minio:9000","aws.s3.region":"us-east-1","aws.s3.use_instance_profile":"false","aws.s3.use_aws_sdk_default_behavior":"false"}
  Enabled: true
  Comment:
1 row in set (0.03 sec)