Supports multiple buckets in a single cluster

[Super-long]

What problem does the new feature solve?

Enables single cluster throughput to be horizontally scalable

What does the feature do?

Supports multiple buckets for object storage in a single cluster

Implementation challenges

No response

[evenyag]

Maybe we can add a name field to the object store config. The ObjectStoreManager uses names to distinguish storages. https://github.com/GreptimeTeam/greptimedb/blob/2d992f4f12c70aaa22477208aee05aa5ee8979e5/src/datanode/src/config.rs#L49-L59

[killme2008]

Maybe we can add a name field to the object store config. The ObjectStoreManager uses names to distinguish storages.

https://github.com/GreptimeTeam/greptimedb/blob/2d992f4f12c70aaa22477208aee05aa5ee8979e5/src/datanode/src/config.rs#L49-L59

yes, that's my plan. Adds a name for storage configs.

[killme2008]

@Super-long Already support it by name in storage.providers, for example

[[storage.providers]]
name = "test1"
type = "S3"
bucket = "test_greptimedb"
root = "/greptimedb"
access_key_id = "<access key id>"
secret_access_key = "<secret access key>"

Then you can create the table with this provider name:

CREATE TABLE IF NOT EXISTS temperatures(
  ts TIMESTAMP TIME INDEX,
  temperature DOUBLE DEFAULT 10,
) engine=mito with(ttl='7d', storage="test1");

This feature will be shipped in the v0.9.3 release.