As we expand or contract the number of zones, we will battle potential co-location issues. I can't tell if the ordering of the Auctioneer optimization is going to handle this correctly or not. Eventually it will but when you are rolling it out which can take however long it takes, the algorithms need to still do the right things with only partial knowledge of the actual world.
@onsi replied:
I agree that this isn't covered in the doc. It's something that I think we can defer so long as we have a plan. As we think about versioning the cluster during rolling deploys I believe that restricting ourselves to unspecified deployment orders is going to give us a big headache.
I think Diego's deployment story gets a bit simpler if we pull the Converger and Auctioneer (both singletons) off of the Cell and onto their own boxes (the Brain?). We'd have N Brains for HA, but only one Brain would ever be active. When performing deploys we'd ensure the Brains roll first.
Now if you increase/decrease AZs the auctioneer will be guaranteed to know before we perform the rolling evacuations. And it should do the right thing.
onsi
commented
9 years ago
@fraenkel asked:
As we expand or contract the number of zones, we will battle potential co-location issues. I can't tell if the ordering of the Auctioneer optimization is going to handle this correctly or not. Eventually it will but when you are rolling it out which can take however long it takes, the algorithms need to still do the right things with only partial knowledge of the actual world.
@onsi replied:
I agree that this isn't covered in the doc. It's something that I think we can defer so long as we have a plan. As we think about versioning the cluster during rolling deploys I believe that restricting ourselves to unspecified deployment orders is going to give us a big headache.
I think Diego's deployment story gets a bit simpler if we pull the Converger and Auctioneer (both singletons) off of the Cell and onto their own boxes (the Brain?). We'd have N Brains for HA, but only one Brain would ever be active. When performing deploys we'd ensure the Brains roll first.
Now if you increase/decrease AZs the auctioneer will be guaranteed to know before we perform the rolling evacuations. And it should do the right thing.
Will open an issue to track this.