search

yugabyte / yugabyte-db

YugabyteDB - the cloud native distributed SQL database for mission-critical applications.
https://www.yugabyte.com
Other
8.99k stars 1.07k forks source link

[Docs] Missing standard tserver removal command #22363

Open frg01 opened 6 months ago

frg01 commented 6 months ago

Description

I have a question, that is what is the standard process for scaling down tserver. I didn't find any standard tserver process in the article. What I know now is that tserver can only be stopped through pkill -15 in Ubuntu, see this picture ,the 192.168.50.138 is Dead,but I cannot see from this page whether 192.168.50.138 has been removed from the cluster. Through my own understanding of the mechanism in the code, I found that the dead status is equivalent to the cluster being removed, but the display on the page seemed to mislead me.

1. The dead information on the page misled me into thinking that tserver is still in the cluster.
2. Why is there no standard command to remove tserver, but the process can only be stopped through pkill -15?

image

At this time, the status of tserver is dead, but it is still in the cluster. It shows that the master must be restarted before it will be removed.

Warning: Please confirm that this issue does not contain any sensitive information

ddorian commented 6 months ago

Hi @frg01

You can remove the servers by using change_blacklist command like in steps 4 & 7 https://docs.yugabyte.com/preview/manage/change-cluster-config/ ?

See https://docs.yugabyte.com/preview/admin/yb-admin/#change-blacklist

frg01 commented 6 months ago

Hi @frg01

You can remove the servers by using change_blacklist command like in steps 4 & 7 https://docs.yugabyte.com/preview/manage/change-cluster-config/ ?

See https://docs.yugabyte.com/preview/admin/yb-admin/#change-blacklist

thanks for your reply ddorian。 I tried it, and the change_blacklists command cannot remove the tserver in the dead state, so this is a bug, right? It is the dead status on the page, which is not in the cluster, but it is displayed together with other tservers in the cluster, causing some misunderstandings.

ajcaldera1 commented 5 months ago

Hi @frg01 -

The appearance of the DEAD server in the tserver list is an artifact of the cached state in the current master leader's memory. The quickest way to refresh this is to elect a new leader. The yb-admin command provides us a subroutine to do exactly that. It is called master_leader_stepdown and it is documented here: https://docs.yugabyte.com/preview/admin/yb-admin/#master-leader-stepdown