Native support for traffic/metrics plugins in Argo Rollouts

Within upstream Argo Rollouts, one can enable traffic management plugins and metric plugins via the standard Argo Rollouts configuration ConfigMap.

However, at present, within Argo Rollouts operator there is no mechanism to specify these via the RolloutManager CR.

As of this writing (once PR #40 has merged), the current operator behaviour is:

If there is an existing traffic management plugin defined in the ConfigMap, it will be preserved when we add the OpenShift GitOps Route plugin (not overwritten)
Any custom metric plugin in the ConfigMap are overwritten (lost).

However, requiring users to modify the ConfigMap to add plugins is not as desirable as our supporting it 'natively', within the RolloutManager CR.

Native support within the RolloutManager CustomResource would look like this:

For example, something like:

kind: RolloutManager
metadata:
  name: my-rollout-install
  namespace: rollouts
spec:
  # (...)
  plugins:
    trafficManagement:
      - name: "argoproj-labs/gatewayAPI"
        location: "https://github.com/argoproj-labs/rollouts-plugin-trafficrouter-gatewayapi/releases/download/<version>/gateway-api-plugin-<arch>" # file url also supported

    metric:
      - name: "argoproj-labs/sample-prometheus"
        location: "https://github.com/argoproj-labs/rollouts-plugin-metric-sample-prometheus/releases/download/v0.0.4/metric-plugin-linux-amd64" # file url also supported
        sha256: "dac10cbf57633c9832a17f8c27d2ca34aa97dd3d" #optional sha256 checksum of the plugin executable

This will cause the specified values to be inserted into the ConfigMap that is generated by the RolloutManager controller.

Open Questions to be discussed and/or investigated by implementor:

Do we need to preserve existing plugins within the ConfigMap, or is it safe to overwrite them now this replacement mechanism is available? My vote is to no longer preserve existing plugins in the ConfigMap.
Do we also need to restart the Rollouts install? Does Rollouts automatically detect the new plugin, once the ConfigMap changes?
- Check whether Rollouts processing plugins on startup, or at any time. If on startup, we will need to restart the Pod.
- Check how Argo CD operator handles similar cases (if they exist)

Issue Criteria:

Add Traffic/metric plugins fields to the RolloutManager custom resource
Modify operator to support new fields: addition/modification/removal of contents of these fields should update the ConfigMap respectively.
Unit tests to verify behaviour as expected: Addition/removal/modification of the fields
E2E tests to verify that Rollouts operator works as expected
- Additional/removal/modification of the fields
- For testing traffic management plugin, we can use the Gateway plugin as an example. An E2E test might do this:
  - 1) Create a RolloutManager pointing to Gateway plugin
  - 2) Wait for RolloutsManager to install Argo Rollouts with the plugin, and wait for the Rollout Deployment to be healthy
  - 3) Create an E2E test that follows the Traefik Gateway API steps: https://github.com/argoproj-labs/rollouts-plugin-trafficrouter-gatewayapi/tree/main/examples/traefik
  - This allows us to verify that the plugin has been successfully loaded, and is running/
- For testing metric plugin, there don't seem to be many plugins that exist. I've found these:
  - https://github.com/argoproj-labs/rollouts-plugin-metric-honeycomb
  - https://github.com/argoproj-labs/rollouts-plugin-metric-sample-prometheus
  - Of these, my preference would be a test case (similar to traffic management plugin) using rollouts-plugin-metric-sample-prometheus

argoproj-labs / argo-rollouts-manager

Native support for traffic/metrics plugins in Argo Rollouts #45