search

ydb-platform / ydb

YDB is an open source Distributed SQL Database that combines high availability and scalability with strong consistency and ACID transactions
https://ydb.tech
Apache License 2.0
4.02k stars 581 forks source link

Split/merge is case of an OVERLOADED datatashard #11849

Open azevaykin opened 11 hours ago

azevaykin commented 11 hours ago

OVERLOADED error is returned by a datashard if:

  1. The partition is not in a normal state, for example, it is being divided/merged.
  2. The input queue of transaction is full (more than 15,000 requests).
  3. The CDC output queue is full (more than 10,000 items or 125 MB).

System view partition_stats has already had counters like InFlightTxCount, TxRejectedByOverload. These metrics can be taken into consideration for split/merge.

kunga commented 11 hours ago

Connected with https://st.yandex-team.ru/KIKIMR-20199

kunga commented 11 hours ago

Maybe DataShard itself should request split without passing dozens of metrics to SchemeShard

And we need a UI button to do the same (split DataShard on request)