Failing test: Dataset quality API Integration tests (basic).x-pack/test/dataset_quality_api_integration/tests/integrations/integrations·spec·ts - Dataset quality API tests integrations/integrations.spec.ts basic Integration gets the installed integrations returns only log based integrations and its datasets map

[kibanamachine]

A test failed on a tracked branch

Error: expected [ 'apm' ] to sort of equal [ 'apm', 'system' ]
    at Assertion.assert (expect.js:100:11)
    at Assertion.eql (expect.js:244:8)
    at Context.<anonymous> (integrations.spec.ts:78:82)
    at processTicksAndRejections (node:internal/process/task_queues:95:5)
    at Object.apply (wrap_function.js:73:16) {
  actual: '[\n  "apm"\n]',
  expected: '[\n  "apm"\n  "system"\n]',
  showDiff: true
}

First failure: kibana-on-merge - main

[elasticmachine]

Pinging @elastic/obs-ux-logs-team (Team:obs-ux-logs)

[yngrdyn]

@elastic/fleet, @elastic/appex-qa could you help us figure out what is happening here? In the build and we are getting the following error

[00:00:23]               │ proc [kibana] [2024-08-02T13:57:32.445+00:00][ERROR][plugins.fleet] '503 Service Temporarily Unavailable' error response from package registry at https://epr.elastic.co/epr/system/system-1.0.0.zip {"service":{"node":{"roles":["background_tasks","ui"]}}}

Looks like something failed regarding package registry, here is another reference where this also happened: https://github.com/elastic/kibana/issues/188676

[kpollich]

Since epr.elastic.co is an internet-facing service exposed over HTTP, it's subject to network flakiness during CI runs. To mitigate this, these tests should be using the dockerized EPR instance available in CI as we do in other suites, e.g.

https://github.com/elastic/kibana/blob/1cb191fb690ddd58333b7c4652f637d0a544da40/x-pack/test/fleet_api_integration/config.base.ts#L45-L57

https://github.com/elastic/kibana/blob/1cb191fb690ddd58333b7c4652f637d0a544da40/x-pack/test/common/services/security_solution/endpoint_registry_helpers.ts#L63-L74

This will spin up a Dockerized instance of EPR available in Kibana CI that will avoid the network hop to epr.elastic.co, and should fix any flakiness associated with these requests. When running locally, you can provide FLEET_PACKAGE_REGISTRY_PORT=12345 as an env var to yarn test:ftr:runner to spin up this container during local FTR runs.

[yngrdyn]

Thanks for your prompt response @kpollich, I'll take a look at the references and accommodate the tests with the dockerized instance.

[kibanamachine]

New failure: kibana-on-merge - 8.15