Incubating Project: DB Mesh For TiDB

[wowdba]

Summary

The high availability and load balancing access to TiDB recommended by PingCAP is solved by load balancing software or hardware such as LVS, HAProxy and F5. Although this solves the problem of load balancing, it brings a layer of network consumption. LVS, one of the more widely used software with Four-layer load balancing, has not been updated for a long time. HAProxy is a software that provides four - and seven-layer load balancing services. but with slightly lower performance than LVS。F5 is a commercial load balancing product with high cost. Therefore, we want to introduce a new Mysql access solution，called DB Mesh，to solve the access problem of TiDB. This project was our in-house developed solution for the MySQL access requirements，but considering TiDB also implements the standard MySQL protocol，we only need small modifications to connect to TiDB, so we decide to open source this project in the form of project incubation, so that more users can enjoy this product and at the same time better tempered products can be provided.

Motivation

The DB Mesh is a access services for cloud-native database (TiDB, MySQL) , mainly for the current mainstream Kubernetes or other virtualized application deployment environment and provides data proxy services by the sidecar way. It provides a grid layer that interacts with the database and it is a zero-intrusion solution for application program, and the problems such as the coupling issue of component and application, complexity of operation and maintenance and network consumption after load balancing can be solved. The product will achieve read-write load balance, read-write separation, process heat update, data source management, SQL detailed acquisition, SQL firewall and other functions with the advantage of the entry point of data traffic.

Labor cost

2 person/month Estimated time required for project delivery It will need 2-3 months for the first phase, mainly focusing on code review, function development, and standardization of existing code to achieve high standard of open source.

Architecture & Deployment

• Architecture

• Deployment

  Core function (Phase I)

Features	Status	Progress
Load balancing	To be verified	Running
Read and write separation	Achieved	Running
Data source configuration management	Need to develop	Running
Connection pool management	Need to develop	Running
Configure hot swap	Achieved	Running
Failover	Achieved	Running
Reporting of monitoring information	Need to develop	Running
Multi-process, multi-version operation on the same port	Achieved	Running
Slow SQL Statistics	Achieved	Running
Daemon and management API interface	Need to develop	Running
Memory OOM management	New feature	Running

[zz-jason]

Maybe we also need user documents for this product?

[zz-jason]

Maybe we also need user documents for this product?

never mind.

This incubating project LGTM 👍

[gregwebs]

The summary explains the problem well. The solution I don't understand :) Is this implementing load balancing at the client?

[tc-frank]

The summary explains the problem well. The solution I don't understand :) Is this implementing load balancing at the client?

Yes， provide load balancing and other many functions（You can refer to the core function introduction above）via independent service process at the client.

[wowdba]

The summary explains the problem well. The solution I don't understand :) Is this implementing load balancing at the client?

Yes.This is a sidecar model.So deployed on the application side. As you said, implementing load balancing at the client.

[winkyao]

LGTM, @gingerkidney @Soline324 Please help @wowdba to incubate this project.

[winkyao]

After discussion, PMC and wowdba still has some concerns about this project, we plan to close it