Migrate Custom threshold rule tests to deployment agnostic solution

[mgiota]

Part of https://github.com/elastic/kibana/issues/183263

🍒 Summary

As part of this PR we introduce a way to write deployment agnostic tests using Mocha tagging. Tests should be written in the new observability_solution_api_integration folder and should be tagged with the @ess and @serverless tags.

describe('@ess @serverless Custom threshold rule, () => {
  describe('Create rule', () => { 
  });
}

Existing tests

Test	Type	ESS	Serverless	Notes
Custom threshold rule type	Alerting	✅	✅	Duplicate tests Serverless: x-pack/test_serverless/api_integration/test_suites/observability/custom_threshold_rule ESS: x-pack/test/alerting_api_integration/observability/custom_threshold_rule

New folder structure

How to migrate

1. create a new custom_threshold folder within alerting folder
2. move custom threshold rule tests there
3. Create serverless.config.ts and ess.config.ts files similar to the burn_rate rule in this PR
4. Add the new config files in .buildkite/ftr_configs.yml
5. Tag the tests with @ess @serverless tags
6. create 2 script commands in the package.json of the observability_solution_api_integration folder similar to the burn_rate commands.
7. For ESS
   • start the server using your new alias alerting_custom_threshold:server:ess
   • start the test runner using the new alias alerting_custom_threshold:runner:ess
8. For Serverless
   • start the server using your new alias alerting_custom_threshold:server:serverless
   • start the test runner using the new alias alerting_custom_threshold:runner:serverless

Notes

It might be that more things need to be done in order to have these tests fully migrated to the new folder structure. For example custom threshold rule tests are currently defined in a feature flag file x-pack/test_serverless/api_integration/test_suites/observability/index.feature_flags.ts

[maryam-saeidi]

Moved to backlog until the related Meta ticket is prioritized.