User with the View Monitoring role is unable to view the list of Active Alerts in the Monitoring UI view

[MKlimuszka]

Rancher Cluster: v2.7.3

Issue description: A user granted the View Monitoring role on the System project in a cluster with rancher-monitoring installed (into the System project), can access Monitoring components via links in the Cluster Explorer Monitoring UI view, but the Active Alerts list in that view is not populated with any alerts (which can be seen by an Administrator).

Using the Browser Developer tools one can observe a HTTP 403 response for a request to k8s/clusters//v1/endpoints/cattle-monitoring-system/rancher-monitoring-alertmanager

Business impact: User with View Monitoring role does not see any on-going active alerts in the Cluster Explorer Monitoring view/misleadingly indicating there are no active alerts.

Troubleshooting steps: N/A

Repro steps:

1. Provision a Rancher v2.7.3 instance and a downstream custom cluster with one all role node and a few workers (to provide sufficient resources to install rancher-monitoring). I used github.com/superseb/tf-do-rancher2)
2. Install rancher-monitoring into the downstream cluster
3. Create a Standard User foo and grant them the View Monitoring role on the downstream cluster's System project and Project Member role on the downstream cluster's Deafult project.
4. Observe that whilst the Active Alerts table in the Monitoring view of the Cluster Explorer is populated for the admin users (i.e. the Watchdog alert is listed), the list is empty for the user foo.
5. Observe HTTP 403 error for k8s/clusters//v1/endpoints/cattle-monitoring-system/rancher-monitoring-alertmanager in browser developer tools' network tab when accessing the Monitoring UI view as the user foo.
6. Create a custom project role granting the get permission on endpoints (no API group restriction) and grant this custom role to the user foo on the System project of the downstream cluster.
7. Observe the user foo can then see the list of Active Alerts successfully populated.

Workaround: Is a workaround available and implemented? Not really What is the workaround: A user could create a custom project role granting the get permission on endpoints, and grant users this permission on the System project; however, this would permit a user access to get all endpoints in System project Namespaces, and not only on rancher-monitoring-alertmanager in the cattle-monitoring-system Namespace.

Actual behavior: User granted 'View Monitoring' role on the System project into which rancher-monitoring is installed is unable to view the list of active Alerts in the Cluster Explorer Monitoring UI view

Expected behavior: User granted 'View Monitoring' role on the System project into which rancher-monitoring is installed is to view the list of active Alerts in the Cluster Explorer Monitoring UI view

Files, logs, traces: N/A

Additional notes: N/A

SURE-6299

[MKlimuszka]

PR is already merged here: https://github.com/rancher/charts/pull/2753

[nickwsuse]

It looks like there isn't an RC cut for the monitoring chart yet, so this isn't quite testable yet.

I was able to mostly repro following the steps in the ticket (also noted below), but after creating a custom project role and allowing get for all endpoints I still wasn't able to view the active alerts as the foo user.

Rancher v2.7-head Commit ID: 95f0b50 && Monitoring Chart v102.0.1+up40.1.2

1. Provision a Rancher v2.7-head instance and a downstream custom cluster with one all role node and three workers
2. Install rancher-monitoring into the downstream cluster
3. Create a Standard User foo and grant them the View Monitoring role on the downstream cluster's System project and Project Member role on the downstream cluster's default project.
4. Observe that whilst the Active Alerts table in the Monitoring view of the Cluster Explorer is populated for the admin users (i.e. the Watchdog alert is listed), the list is empty for the foo user.
5. Observe HTTP 403 error for k8s/clusters//v1/endpoints/cattle-monitoring-system/rancher-monitoring-alertmanager in browser developer tools' network tab when accessing the Monitoring UI view as the user foo.
6. Create a custom project role granting the get permission on endpoints (no API group restriction) and grant this custom role to the foo user on the System project of the downstream cluster.
7. Observe the foo user can then see the list of Active Alerts successfully populated.

One thing I noticed in my attempts to reproduce the issue is that that the foo user can access the Active Alerts if they click into any of the other monitoring chart options and then back into the monitoring dashboard...

[prachidamle]

Moving to-test since the charts version update PR has merged.

[nickwsuse]

Verified on v2.7-head Commit ID: 1955476

The Active Alerts are now viewable with the View Monitoring role for non-admin users.

1. Fresh Install - Pass
2. Rancher Upgrade - Pass
3. Chart Upgrade - Pass