Release states and strategy conditions not updated when the Cluster Client Store can't provide a client

[parhamdoustdar]

When Shipper can't connect to an app cluster, the strategy field of the release doesn't get updated at all. To reproduce, create a release, wait for it to reach step 0, change the apiMaster of the Cluster object so that it's invalid, and patch the targetStep to 1. Then, you should see something like this:

$ kubectl get rel -o json | jq '.items[0].status.conditions'
[
  {
    "lastTransitionTime": "2020-06-04T09:44:39Z",
    "status": "False",
    "type": "Blocked"
  },
  {
    "lastTransitionTime": "2020-06-04T09:44:39Z",
    "message": "kind-app",
    "reason": "ClustersChosen",
    "status": "True",
    "type": "ClustersChosen"
  },
  {
    "lastTransitionTime": "2020-06-04T12:08:05Z",
    "message": "cluster \"kind-app\" not ready for use yet; cluster client is being initialized",
    "reason": "StrategyExecutionFailed",
    "status": "False",
    "type": "StrategyExecuted"
  }
]

Now, if you look at the strategy field, you will see no trace of this error:

$ kubectl get rel -o json | jq '.items[0].status.strategy'
{
  "clusters": [
    {
      "conditions": [
        {
          "lastTransitionTime": "2020-06-04T11:47:33Z",
          "message": "",
          "reason": "",
          "status": "True",
          "step": 0,
          "type": "ContenderAchievedCapacity"
        },
        {
          "lastTransitionTime": "2020-06-04T11:47:33Z",
          "message": "",
          "reason": "",
          "status": "True",
          "step": 0,
          "type": "ContenderAchievedInstallation"
        },
        {
          "lastTransitionTime": "2020-06-04T11:47:33Z",
          "message": "",
          "reason": "",
          "status": "True",
          "step": 0,
          "type": "ContenderAchievedTraffic"
        }
      ],
      "name": "kind-app"
    }
  ],
  "state": {
    "waitingForCapacity": "False",
    "waitingForCommand": "True",
    "waitingForInstallation": "False",
    "waitingForTraffic": "False"
  }
}

Note that:

• The conditions say that the contender has achieved capacity and traffic, which is not true
• The states say that the Release is waiting for command, which is not true