search

tidb-incubator / Serverlessdb-for-HTAP

a serverless db service based on TiDB on the cloud, implementing dynamically scale the compute and storage nodes based on business load changes.
Apache License 2.0
29 stars 12 forks source link

[Feature] Users will not be affected at all when auto-scaling #7

Open renajin opened 3 years ago

renajin commented 3 years ago

Feature Request

Is your feature request related to a problem? Please describe: When the middleware decides to scale-in, there are still some running sqls at this time, if the tidb node currently running sql is deleted, connection interrupted is a terrible thing.

Describe the feature you'd like: Ensure that these processes are not affected.

Describe alternatives you've considered: We use admission webhook to solve this problem, scale module decide which nodes will be deleted and inform nodes' indexs to statefulset, the validating admission webhook now can intercept this request and distinguish whether it is tidb or other resources like tikv,pd, and then, send request of tidb to proxy with deletion information, The proxy will forward traffic of those nodes to other tidbs and ensure that sql being executed ends smoothly, when there are no more unprocessed tasks on these nodes, the proxy will notice webhook to release that deletion request, Finally statefulset will safely delete those nodes

Teachability, Documentation, Adoption, Migration Strategy:

spxcmss commented 3 years ago

Which nodes are selected to be deleted based on which indicators?

renajin commented 3 years ago

By calculating the number of transactions, scale module can select nodes with the smaller number of transactions, and then advanced statefulSet will delete those nodes.