[Cloud Security] CSPM for GCP - Preview API request shows a misleading preview

[amirbenun]

Kibana version: 8.12.0-SNAPSHOT

Elasticsearch version: 8.12.0-SNAPSHOT

Server OS version:

Browser version:

Browser OS version:

Original install method (e.g. download page, yum, from source, etc.):

Describe the bug: Preview API request doesn't work well on the installation of CSPM integration for GCP. The installation of CSPM for GCP integration for "single account" with "manual" setup works well when browsing kibana. However, the preview API request shows a preview which is not exactly the same as the actual request sent by kibana. This gap doesn't cause any harm on other CSPM installations but just in the flow I described (GCP manual setup).

Preview API request

```json POST kbn:/api/fleet/package_policies { "policy_id": "agentless", "package": { "name": "cloud_security_posture", "version": "1.6.5", "experimental_data_stream_features": [] }, "name": "cspm-1", "description": "", "namespace": "default", "inputs": { "kspm-cloudbeat/cis_k8s": { "enabled": false, "streams": { "cloud_security_posture.findings": { "enabled": false } } }, "kspm-cloudbeat/cis_eks": { "enabled": false, "streams": { "cloud_security_posture.findings": { "enabled": false, "vars": {} } } }, "cspm-cloudbeat/cis_aws": { "enabled": false, "streams": { "cloud_security_posture.findings": { "enabled": false, "vars": { "aws.credentials.type": "cloud_formation", "aws.account_type": "organization-account" } } } }, "cspm-cloudbeat/cis_gcp": { "enabled": true, "streams": { "cloud_security_posture.findings": { "enabled": true, "vars": { "gcp.account_type": "single-account", "setup_access": "manual" } } } }, "cspm-cloudbeat/cis_azure": { "enabled": false, "streams": { "cloud_security_posture.findings": { "enabled": false } } }, "vuln_mgmt-cloudbeat/vuln_mgmt_aws": { "enabled": false, "streams": { "cloud_security_posture.vulnerabilities": { "enabled": false } } } }, "vars": { "posture": "cspm", "deployment": "gcp" } } ```

When running the preview request it returns the following:

{
  "statusCode": 400,
  "error": "Bad Request",
  "message": "Variable cspm-cloudbeat/cis_gcp cloud_security_posture.findings:setup_access not found"
}

Steps to reproduce:

1. Install the CSPM integration
2. Choose GCP, single account, manual setup
3. Click "Preview API request" and try to run it in the console

Expected behavior:

Screenshots (if relevant):

Errors in browser console (if relevant):

Provide logs and/or server output (if relevant):

Any additional context: FYI @elastic/kibana-cloud-security-posture

[maxcold]

@amirbenun, @oren-zohar mentioned that this issue affects the QA flow in some way. Can you elaborate on what is the problem there to properly prio the ticket? From the description, it only looks like a problem with Preview API which doesn't affect anything except if a user decides to use the Preview API output somehow. But if it affects our QA cycle or QA github actions - then it's a higher prio ofc

[amirbenun]

The "Create Environment" automated job installs the integrations using API calls. @gurevichdmitry recently added an infrastructure that generates requests based on the preview request from Kibana. In most cases it works as expected, it fails just with GCP configured as I mentioned above. Therefore, the ideal fix for us would be to add support for that request structure as it appears on the preview (attached to the issue).