[8.17] [APM] Migrate settings API tests to be deployment-agnostic (#200762)

[iblancof]

Backport

This will backport the following commits from main to 8.17:

• [APM] Migrate settings API tests to be deployment-agnostic (#200762)

Questions ?

Please refer to the Backport tool documentation

\r\n\r\n---------\r\n\r\nCo-authored-by: Carlos Crespo \r\nCo-authored-by: Carlos Crespo ","sha":"05bf56f3365e9f273bad6d29cc5855d9c3607fc7","branchLabelMapping":{"^v9.0.0$":"main","^v8.18.0$":"8.x","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:skip","v9.0.0","apm","backport:prev-minor","ci:project-deploy-observability","Team:obs-ux-infra_services","v8.17.0"],"number":200762,"url":"https://github.com/elastic/kibana/pull/200762","mergeCommit":{"message":"[APM] Migrate settings API tests to be deployment-agnostic (#200762)\n\nCloses https://github.com/elastic/kibana/issues/198989\r\nPart of https://github.com/elastic/kibana/issues/193245\r\n\r\nThis PR contains the changes to migrate `settings` test folder to\r\ndeployment-agnostic testing strategy.\r\n\r\n\r\n**Not Migrated**\r\n- `agent_configuration`: Not available in Serverless.\r\n- `anomaly_detection/no_access`: Involves the `noAccess` user role; we\r\nare only migrating tests for `viewer`, `editor`, or `admin` roles.\r\n- `anomaly_detection/update_to_v3`: Involves the deletion of ML jobs; we\r\nwill wait until an \"ml\" service is available to properly migrate these\r\ntests.\r\n- `anomaly_detection/write_user`: Involves the deletion of ML jobs; we\r\nwill wait until an \"ml\" service is available to properly migrate these\r\ntests.\r\n\r\n**Partially Migrated**\r\n- `anomaly_detection/read_user`: Involves the\r\n`apmAllPrivilegesWithoutWriteSettingsUser` role; only tests for the\r\n`read` role have been migrated.\r\n- `anomaly_detection/write_user`: Involves the\r\n`apmReadPrivilegesWithWriteSettingsUser` role; only tests for the\r\n`write` role have been migrated.\r\n- `apm_indices`: Tests based on license have not been migrated.\r\ncustom_link: Involves the `apmReadPrivilegesWithWriteSettingsUser` role;\r\nonly tests for the trial `write` role have been migrated.\r\n- `agent_keys`: Involves the `manageOwnAgentKeysUser` and\r\n`createAndAllAgentKeysUser` roles; only tests for the `write` role have\r\nbeen migrated.\r\n\r\n### How to test\r\n\r\n\r\n- Serverless\r\n\r\n```\r\nnode scripts/functional_tests_server --config x-pack/test/api_integration/deployment_agnostic/configs/serverless/oblt.apm.serverless.config.ts\r\nnode scripts/functional_test_runner --config x-pack/test/api_integration/deployment_agnostic/configs/serverless/oblt.apm.serverless.config.ts \r\n```\r\n\r\nIt's recommended to be run against\r\n[MKI](https://github.com/crespocarlos/kibana/blob/main/x-pack/test_serverless/README.md#run-tests-on-mki)\r\n\r\n- Stateful\r\n```\r\nnode scripts/functional_tests_server --config x-pack/test/api_integration/deployment_agnostic/configs/stateful/oblt.apm.stateful.config.ts \r\nnode scripts/functional_test_runner --config x-pack/test/api_integration/deployment_agnostic/configs/stateful/oblt.apm.stateful.config.ts \r\n```\r\n\r\n## Checks\r\n\r\n- [ ] (OPTIONAL, only if a test has been unskipped) Run flaky test suite\r\n- [x] local run for serverless\r\n- [x] local run for stateful\r\n- [x] MKI run for serverless \r\n\r\n\r\n\r\n---------\r\n\r\nCo-authored-by: Carlos Crespo \r\nCo-authored-by: Carlos Crespo ","sha":"05bf56f3365e9f273bad6d29cc5855d9c3607fc7"}},"sourceBranch":"main","suggestedTargetBranches":["8.17"],"targetPullRequestStates":[{"branch":"main","label":"v9.0.0","labelRegex":"^v9.0.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/200762","number":200762,"mergeCommit":{"message":"[APM] Migrate settings API tests to be deployment-agnostic (#200762)\n\nCloses https://github.com/elastic/kibana/issues/198989\r\nPart of https://github.com/elastic/kibana/issues/193245\r\n\r\nThis PR contains the changes to migrate `settings` test folder to\r\ndeployment-agnostic testing strategy.\r\n\r\n\r\n**Not Migrated**\r\n- `agent_configuration`: Not available in Serverless.\r\n- `anomaly_detection/no_access`: Involves the `noAccess` user role; we\r\nare only migrating tests for `viewer`, `editor`, or `admin` roles.\r\n- `anomaly_detection/update_to_v3`: Involves the deletion of ML jobs; we\r\nwill wait until an \"ml\" service is available to properly migrate these\r\ntests.\r\n- `anomaly_detection/write_user`: Involves the deletion of ML jobs; we\r\nwill wait until an \"ml\" service is available to properly migrate these\r\ntests.\r\n\r\n**Partially Migrated**\r\n- `anomaly_detection/read_user`: Involves the\r\n`apmAllPrivilegesWithoutWriteSettingsUser` role; only tests for the\r\n`read` role have been migrated.\r\n- `anomaly_detection/write_user`: Involves the\r\n`apmReadPrivilegesWithWriteSettingsUser` role; only tests for the\r\n`write` role have been migrated.\r\n- `apm_indices`: Tests based on license have not been migrated.\r\ncustom_link: Involves the `apmReadPrivilegesWithWriteSettingsUser` role;\r\nonly tests for the trial `write` role have been migrated.\r\n- `agent_keys`: Involves the `manageOwnAgentKeysUser` and\r\n`createAndAllAgentKeysUser` roles; only tests for the `write` role have\r\nbeen migrated.\r\n\r\n### How to test\r\n\r\n\r\n- Serverless\r\n\r\n```\r\nnode scripts/functional_tests_server --config x-pack/test/api_integration/deployment_agnostic/configs/serverless/oblt.apm.serverless.config.ts\r\nnode scripts/functional_test_runner --config x-pack/test/api_integration/deployment_agnostic/configs/serverless/oblt.apm.serverless.config.ts \r\n```\r\n\r\nIt's recommended to be run against\r\n[MKI](https://github.com/crespocarlos/kibana/blob/main/x-pack/test_serverless/README.md#run-tests-on-mki)\r\n\r\n- Stateful\r\n```\r\nnode scripts/functional_tests_server --config x-pack/test/api_integration/deployment_agnostic/configs/stateful/oblt.apm.stateful.config.ts \r\nnode scripts/functional_test_runner --config x-pack/test/api_integration/deployment_agnostic/configs/stateful/oblt.apm.stateful.config.ts \r\n```\r\n\r\n## Checks\r\n\r\n- [ ] (OPTIONAL, only if a test has been unskipped) Run flaky test suite\r\n- [x] local run for serverless\r\n- [x] local run for stateful\r\n- [x] MKI run for serverless \r\n\r\n\r\n\r\n---------\r\n\r\nCo-authored-by: Carlos Crespo \r\nCo-authored-by: Carlos Crespo ","sha":"05bf56f3365e9f273bad6d29cc5855d9c3607fc7"}},{"branch":"8.17","label":"v8.17.0","labelRegex":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"}]}] BACKPORT-->

[elasticmachine]

Pinging @elastic/obs-ux-infra_services-team (Team:obs-ux-infra_services)

[github-actions[bot]]

:robot: GitHub comments

Expand to view the GitHub comments

Just comment with: - `/oblt-deploy` : Deploy a Kibana instance using the Observability test environments. - `run` `docs-build` : Re-trigger the docs validation. (use unformatted text in the comment!)

[elasticmachine]

:yellow_heart: Build succeeded, but was flaky

• Buildkite Build
• Commit: fda66a9dc438e85a8ae6142637f4341d1bd8deb4

Failed CI Steps

• FTR Configs #20

Test Failures

• [job] [logs] FTR Configs #20 / endpoint Endpoint permissions: when running with user/role [t1_analyst] should display endpoint data on Host Details

Metrics [docs]

✅ unchanged

cc @iblancof