[Cloud Security] [Agentless] Include X-Request-ID header in all Agentless API requests and logs

[opauloh]

Motivation

Follow-up from https://github.com/elastic/agentless-api/issues/242, this ticket aims to include the X-Request-ID Header on all HTTP calls from the Kibana server to the Agentless API.

The idea of the X-Request-ID is that Kibana can create some random ID and pass it to the Agentless API. Both Kibana and the Agentless API will then include that ID in every log statement that it creates (creating agents, deleting agents, general errors, etc...).

If a customer receives an error it can include the ID in a bug report, allowing the SDH operators to look up the corresponding log statements (without having to rely on timestamps, IPs, etc).

As this ID is generated (randomly) by Kibana it does not contain any sensitive information (client_id, cluster_id, etc), and should not violate the user's privacy as a unique ID is created per request it does also not help with tracking users but provides insights for debugging issues.

Additional Info

For storing the correlation ID in Elasticsearch documents, ECS suggests using the http.request.id field.

Definition of done

• [ ] X-Request-ID Header included on all HTTP calls from the Kibana server to the Agentless API.
• [ ] X-Request-ID is generated (randomly) and does not contain any sensitive information.
• [ ] X-Request-ID is unique per request.
• [ ] X-Request-ID is included on the error logs thrown on the try/catch blocks.
• [ ] If any logging document is created on Elasticsearch it should use the http.request.id field to store the value of X-Request-ID.

• Modifications on the agentless API TBD in a separate ticket.

Related tasks/epics

• Discussion

[elasticmachine]

Pinging @elastic/kibana-cloud-security-posture (Team:Cloud Security)

[opauloh]

Closing this ticket in favour of https://github.com/elastic/kibana/issues/192048