Proxied resources don't properly handle trusted header authentication

[thll]

Expected Behavior

A call to a proxied resource, like /api/cluster/metrics/multiple should properly handle trusted header authentication.

Current Behavior

When the client session expires but trusted headers are used, the call to /api/cluster/metrics/multiple is authenticated, but the cluster-internal subsequent calls to /api/system/metrics/multiple don't provide any authentication data. They are therefore rejected as unauthorized by the receiving nodes.

This leads to the following effect: When Trusted Header Authentication is enabled, and the session of a user expires, the frontend is still usable. But the global throughput counter drops to zero. Only after the page is reloaded and a new session cookie is included in the requests the throughput counter becomes functional again.

Possible Solution

• We could try to fix the frontend to not send any requests to any resources other than those for session creation.
• Also, we could fail early in the proxied resource if the original request was authenticated, but we are unable to add authentication information to the federated requests here

Steps to Reproduce (for bugs)

1. Set up a Graylog cluster of at least two nodes
2. Set up a local Graylog user with a short session expiration (e.g. use 30 seconds)
3. Set up Trusted Header Authentication and make sure to set the header to the Graylog user for every request. Use a proxy or a browser plugin to do that.
4. Open the UI which should log you in automatically. Wait for 30 seconds until the session times out.
5. Notice errors similar to the following in the server.log:

   2022-10-18 15:41:43,596 WARN : org.graylog2.shared.rest.resources.ProxiedResource - Unable to call http://127.0.0.1:9001/api/system/metrics/multiple on node <fd4d8b06-ad11-46e7-ae0b-78a611c56311>, result: Unauthorized
   2022-10-18 15:41:43,597 WARN : org.graylog2.shared.rest.resources.ProxiedResource - Unable to call http://127.0.0.1:9000/api/system/metrics/multiple on node <9c0c55fa-548f-4a44-9b8f-1138b3c5c4c3>, result: Unauthorized

Context

[HS-1127509680]

Your Environment

• Graylog Version: master
• Java Version:
• Elasticsearch Version:
• MongoDB Version:
• Operating System:
• Browser version:

[williamtrelawny]

Original customer who reported this issue in 4.3/5.0 seems to be experiencing this again in 5.1.

ref: HS-1939510548