Tracking issue for the MetricEngine

[waynexia]

What type of enhancement is this?

Performance

What does the enhancement do?

RFC metric-engine proposes a new region engine Metric. Here is the tracking issue for implementing it.

Related progress is also tracked in GitHub Project https://github.com/orgs/GreptimeTeam/projects/34

• [x] RFC document https://github.com/GreptimeTeam/greptimedb/pull/1925
• Meta Server
  • [x] Reserve the first 8 bits of region number for region id group https://github.com/GreptimeTeam/greptimedb/pull/2100
  • [ ] Add logical route entry to table route
  • [ ] Add special logic to create progress when table engine is metric
  • [x] New alter procedure that acquires the physical table's resource lock before operating the logical table https://github.com/GreptimeTeam/greptimedb/pull/3098
  • [ ] Support grouping CREATE requests
  • [ ] Translate table name of physical table to its table id when handle CREATE request of logical table
  • [ ] Support alter to logical table
• Frontend
  • [ ] Also support automatically creating the physical table under public schema.
  • [ ] Convert requests from prometheus remote write into one request to the physical table
  • [ ] Generate DDL for logical table from incoming prom remote write requests and submit to metasrv (for updating metadata of logical table)
  • [x] Separate GreptimeDatabase service into two different services
• Datanode
  • [x] Define RegionEngine and RegionServer #2160
  • [x] Define MetricEngine on top of MitoEngine
  • [x] Define MetadataRegion to handle operations to metadata #2687
  • [x] DataRegion to handle operations to data https://github.com/GreptimeTeam/greptimedb/pull/2706
  • [x] Handle Create request https://github.com/GreptimeTeam/greptimedb/pull/2694
  • [x] Handle Write request https://github.com/GreptimeTeam/greptimedb/pull/2759
  • [x] Handle Read request https://github.com/GreptimeTeam/greptimedb/pull/2793
  • [x] Handle Alter request https://github.com/GreptimeTeam/greptimedb/pull/2726
  • [x] Forward INSERT/SELECT to physical table to mito engine (is implemented together with corresponding logical operation)
  • [ ] Be able to recognize physical table (by their table id?)
  • [ ] Implement failover/heartbeat/... for MetricEngine
  • [ ] Change a platform-independent hash algorithm for TSID
  • [ ] Maintains the schema of physical table in metasrv

Implementation challenges

No response

[tisonkun]

Is the TODO list above updated? Perhaps we can create subtasks issue for each task so that we can keep the status synced.

Take https://docs.github.com/en/get-started/writing-on-github/working-with-advanced-formatting/about-task-lists as the reference.

[killme2008]

From now on, we need to keep compatibility with older versions as possible as we can.