search

elastic / kibana

Your window into the Elastic Stack
https://www.elastic.co/products/kibana
Other
19.48k stars 8.04k forks source link

[Fleet] Enterprise Search plugins are not included in the total count of custom integrations on CI #115108

Open yakhinvadim opened 2 years ago

yakhinvadim commented 2 years ago

cc: @thomasneirynck

Integrations tests are passing when being run with 2 functional test runner commands (server and runner) and failing when being run with 1 combined command, which is used on CI.

Discovered in this PR: https://github.com/elastic/kibana/pull/114919#issuecomment-943709702

To reproduce:

  1. Add 18 (number of Workplace Search integrations) to the expected number on this line: https://github.com/elastic/kibana/blob/e49ec25fae79ed83245d23cf33edcdaaaf912fcd/test/api_integration/apis/custom_integration/integrations.ts#L26

  2. Run the following commands from the x-pack folder.

    node scripts/functional_tests_server

    And in a separate terminal:

    node ../scripts/functional_test_runner --config ../test/api_integration/config.js --grep="customIntegrations"

    Expected result: 2/2 tests are passing.

  3. Close previously opened terminals and run the command from CI from the kibana root:

    node scripts/functional_tests --config test/api_integration/config.js --bail --debug

    Expected result: all tests are passing Actual result: 1 functional test failure. Error: expected 15 to be above 32

elasticmachine commented 2 years ago

Pinging @elastic/fleet (Team:Fleet)

joshdover commented 2 years ago

I believe this is happening because those integration tests are not run with x-pack plugins enabled, which is a test environment issue and not a product bug