Expand non-voting replicas architecture docs to explain how they work with each multi-region table locality

[jseldess]

Jesse Seldess (jseldess) commented:

When reading the great non-voting replicas docs for 21.1, this statement provoked some questions for me:

In order to provide better support for multi-region clusters, (including the features that make fast multi-region reads possible), a new type of replica is introduced: the non-voting replica.

"fast multi-region reads" links to the GLOBAL table locality pattern, which make sense. From what's written here and in the non-blocking transactions docs, I understood the relation of non-voting replicas to GLOBAL. However, the language above is not exclusive to that pattern. That led me to wonder how non-voting replicas are used for the REGIONAL patterns.

I asked @ajstorm and he clarified that we create non-voting replicas for regional patters, in the non-primary regions, by default, too, as long as a region doesn't already have a voting replica, which, from what I understand, can be the impact of survivability goals (whoa!).

Although I understand and agree with the desire to keep our multi-region explanation and guidance simple and intuitive for the vast majority of users, I think we need to go further in explaining the mechanics in our architecture docs, for both internal and external power users who want/need to go there. I know this isn't a priority for 21.1 GA, but I'd like it to be considered for the future. It goes hand-in-hand with updating the way we describe and visualize our replication and transaction models in presentations (and maybe Cockroach U).

The only thought I have for the multi-region user guidance (as opposed to architecture reference) is whether we should somehow call out that the regional patterns also get you good reads from non-primary regions. Right now, that's not emphasized anywhere. Is there a way to do that without adding too much complexity?

cc @rmloveland, @awoods187, @ajstorm

Jira Issue: DOC-1614

[awoods187]

We will definitely want to cover doing follower reads from other regions of partitioned data (the reason we added non-voting replicas) as an enhancement to the existing tutorials. I'm supportive of including more detail in the architecture docs but as you noted its a bit lower on the priority for the moment.

[github-actions[bot]]

We have marked this issue as stale because it has been inactive for 18 months. If this issue is still relevant, removing the stale label or adding a comment will keep it active. Otherwise, we'll close it in 10 days to keep the issue queue tidy. Thank you for your contribution to CockroachDB docs!