search

apple / foundationdb

FoundationDB - the open source, distributed, transactional key-value store
https://apple.github.io/foundationdb/
Apache License 2.0
14.47k stars 1.31k forks source link

Add finer grained controls of data distribution #1508

Open ajbeamon opened 5 years ago

ajbeamon commented 5 years ago

There exists a command in fdbcli to disable data distribution, and it disables all data movement. Often this isn't quite what we want, though. For example, doing this prevents the cluster from managing hot write shards, leading to a saturated cluster.

Instead, we should have the ability to control at least the following types of data movement independently:

Failures Splitting/merging Rebalancing

There may also be some value in being able to allow movement that heals replication factor but not movement that deals with unhealthy states that have full replication.

dongxinEric commented 5 years ago

This has been resolved.

ajbeamon commented 5 years ago

@dongxinEric Does the PR address all 4 of the cases above?

dongxinEric commented 5 years ago

@dongxinEric Does the PR address all 4 of the cases

I forgot to update this issue about that after some discussion with Evan, the PRs merged only added support for Failures and Rebalancing. Let me reopen this since the splitting and merging part is still not supported via the CLI.

ajbeamon commented 5 years ago

Sounds good. There's also the question of whether we want to be able to control movement for unhealthy teams that doesn't repair replication health.

ajbeamon commented 4 years ago

@etschannen Do we still want to add the ability to control the other movement types?

etschannen commented 4 years ago

I do not think there is a reason to stop splits and merges, which is why I closed it. If you think there is potentially a reason we would want to do that we could reopen it.

ajbeamon commented 4 years ago

@etschannen there is the fourth one for stopping movement dealing with unhealthy teams that have full replication, and also maybe the team remover stuff is relevant too now. I think I would still like to see splits and merges supported too, especially once were no longer using the mutation log.