During meta recovery ru.updateConsumers and ru.removeConsumers would not be properly cleared since the move from map[string]*consumerAssignment to map[string]map[string]*consumerAssignment. Which meant that consumers that needed to be removed were both in ru.removeConsumers and left in ru.updateConsumers. Resulting in a ghost consumer.
Also don't clear recovering state while we still have items to process as part of recovery.
De-flakes TestJetStreamClusterLostConsumers, and makes TestJetStreamClusterConsumerLeak more reliable by re-introducing the ca.pending flag. Since the consumer leader responds for consumer creation, but meta leader responds for consumer deletion, so need to have the consumer assignment available so meta leader can respond successfully.
During meta recovery
ru.updateConsumersandru.removeConsumerswould not be properly cleared since the move frommap[string]*consumerAssignmenttomap[string]map[string]*consumerAssignment. Which meant that consumers that needed to be removed were both inru.removeConsumersand left inru.updateConsumers. Resulting in a ghost consumer.Also don't clear recovering state while we still have items to process as part of recovery.
De-flakes
TestJetStreamClusterLostConsumers, and makesTestJetStreamClusterConsumerLeakmore reliable by re-introducing theca.pendingflag. Since the consumer leader responds for consumer creation, but meta leader responds for consumer deletion, so need to have the consumer assignment available so meta leader can respond successfully.Signed-off-by: Maurice van Veen github@mauricevanveen.com