Apache Doris Manager 2022 Roadmap

[hf200012]

Doris Manager version 1.0 will be released at the end of February. This version can realize one-click visualization and automatic installation and deployment, start and stop, configuration management, etc. of Doris clusters. In this way, Doris Manager basically realizes the visual installation, deployment, and management of clusters, which greatly reduces the difficulty of using Doris for new users. For the subsequent development, our overall plan is as follows, and we will send an email for discussion.

I need everyone to discuss together to decide the follow-up function development of Doris Manager, and collect everyone's opinions and suggestions

The overall plan is as follows：

• Doris stock cluster takeover

  • [ ] Existing cluster takeover (implementing takeover of installed clusters, including cluster configuration modification, startup, monitoring, etc.)

• Doris visual upgrade support

• Cluster node list management

  • [ ] Cluster overview view: such as node book, FE, BE, etc., running time, resource usage, number of running Load jobs, etc.
  • [ ] The current cluster list UI is optimized, refer to Ambari, change it to the machine list, and display the roles by tab (FE, BE, Broker)
  • [ ] You can click on the node to view the information of this node, including the relevant information of the node machine, installed services, node configuration information (BE, FE), node monitoring information (memory, disk, CPU, network, load, etc.)
    • [ ] On a specific node, the services on this node can be started and stopped, etc.
    • [ ] Can visualize elastic scaling clusters

• Cluster monitoring

  • [ ] FE, BE running status monitoring and warning
  • [ ] Load job monitoring alert
  • [ ] FE JVM monitoring and warning
  • [ ] Cluster FE JVM heap statistics
  • [ ] Cluster BE memory usage overview
  • [ ] Max Replayed journal id monitoring and warning
  • [ ] BDBJE Write
  • [ ] Tablet scheduling
  • [ ] BE IO statistics
  • [ ] BE Compaction Score monitoring warning
  • [ ] Query Statistic The number of query requests and response time in this part
  • [ ] BE BC (Base Compaction) and CC (Compaction Cumulate) monitoring and warning, etc.

• Doris plugin support

  • [ ] Automatic installation and deployment of the audit log plug-in, realize the automatic installation of the audit log plug-in, and modify the configuration information of the plug-in at the same time.
  • [ ] Create a plug-in database, create a data table, and present the installation information of the plug-in, mainly the database and table information where the plug-in collects audit logs.
  • [ ] Give the query and display page of the audit log

• Gradually weaken the current Doris web UI

  • [ ] Management of Doris cluster users, roles and permissions can be achieved through Doris Manager
  • [ ] Provides a SQL development console with associative input
  • [ ] Provides a visual display of SQL Profile, similar to HUE, which helps users analyze SQL performance
  • [ ] The SQL console supports saving queries for subsequent reuse after use, saving query history, etc.
  • [ ] The above function is also applicable to the analysis of the sql list on the current Web UI

[maninhill]

Is there any progress on this Doris visualization management tool?

[JusticeGong]

Is there any progress on this Doris visualization management tool?

Recently we've done a lot progress on Doris Manager. The first version Doris Manager 1.0 is almost finished and will be released in recent days. We will try to provide a precompiled version to community in next week and all guys are welcome to test and contribute. Doris Manager 1.0 mainly focus on cluster deployment and configuration. Users can deploy a new cluster via visualized process, or connect the existing cluster to Manager.

[gj-zhang]

When will it be released? I can't wait. Which branch or repository can compile?

[morningman]

The Doris Manager Repo has been moved to https://github.com/apache/incubator-doris-manager

[hf200012]

The Doris Manager Repo has been moved to https://github.com/apache/incubator-doris-manager

OK