Failing test: X-Pack Search Sessions Integration.x-pack/test/search_sessions_integration/tests/apps/dashboard/async_search/save_search_session_relative_time·ts - Dashboard save a search sessions with relative time Saves and restores a session with relative time ranges

kibanamachine commented 3 years ago

A test failed on a tracked branch

Error: retry.try timeout: Error: expected 9 to equal 11
    at Assertion.assert (/dev/shm/workspace/parallel/4/kibana/node_modules/@kbn/expect/expect.js:100:11)
    at Assertion.be.Assertion.equal (/dev/shm/workspace/parallel/4/kibana/node_modules/@kbn/expect/expect.js:227:8)
    at Assertion.be (/dev/shm/workspace/parallel/4/kibana/node_modules/@kbn/expect/expect.js:69:22)
    at /dev/shm/workspace/parallel/4/kibana/test/functional/services/dashboard/expectations.ts:210:43
    at runMicrotasks (<anonymous>)
    at processTicksAndRejections (internal/process/task_queues.js:93:5)
    at runAttempt (/dev/shm/workspace/parallel/4/kibana/test/common/services/retry/retry_for_success.ts:27:15)
    at retryForSuccess (/dev/shm/workspace/parallel/4/kibana/test/common/services/retry/retry_for_success.ts:66:21)
    at Retry.try (/dev/shm/workspace/parallel/4/kibana/test/common/services/retry/retry.ts:32:14)
    at DashboardExpect.savedSearchRowCount (/dev/shm/workspace/parallel/4/kibana/test/functional/services/dashboard/expectations.ts:205:7)
    at checkSampleDashboardLoaded (test/search_sessions_integration/tests/apps/dashboard/async_search/save_search_session_relative_time.ts:96:5)
    at Context.<anonymous> (test/search_sessions_integration/tests/apps/dashboard/async_search/save_search_session_relative_time.ts:68:7)
    at Object.apply (/dev/shm/workspace/parallel/4/kibana/node_modules/@kbn/test/src/functional_test_runner/lib/mocha/wrap_function.js:73:16)
    at onFailure (/dev/shm/workspace/parallel/4/kibana/test/common/services/retry/retry_for_success.ts:17:9)
    at retryForSuccess (/dev/shm/workspace/parallel/4/kibana/test/common/services/retry/retry_for_success.ts:57:13)
    at Retry.try (/dev/shm/workspace/parallel/4/kibana/test/common/services/retry/retry.ts:32:14)
    at DashboardExpect.savedSearchRowCount (/dev/shm/workspace/parallel/4/kibana/test/functional/services/dashboard/expectations.ts:205:7)
    at checkSampleDashboardLoaded (test/search_sessions_integration/tests/apps/dashboard/async_search/save_search_session_relative_time.ts:96:5)
    at Context.<anonymous> (test/search_sessions_integration/tests/apps/dashboard/async_search/save_search_session_relative_time.ts:68:7)
    at Object.apply (/dev/shm/workspace/parallel/4/kibana/node_modules/@kbn/test/src/functional_test_runner/lib/mocha/wrap_function.js:73:16)

First failure: Jenkins Build

elasticmachine commented 3 years ago

Pinging @elastic/kibana-presentation (Team:Presentation)

tylersmalley commented 3 years ago

Currently only three failures in the past week:

kibanamachine commented 3 years ago

New failure: Jenkins Build

kibanamachine commented 3 years ago

New failure: Jenkins Build

kibanamachine commented 3 years ago

New failure: Jenkins Build

kibanamachine commented 3 years ago

New failure: Jenkins Build

kibanamachine commented 3 years ago

New failure: Jenkins Build

kibanamachine commented 3 years ago

New failure: Jenkins Build

kibanamachine commented 3 years ago

New failure: Jenkins Build

kibanamachine commented 3 years ago

New failure: Jenkins Build

kibanamachine commented 3 years ago

New failure: Jenkins Build

kibanamachine commented 3 years ago

New failure: Jenkins Build

kibanamachine commented 3 years ago

New failure: Jenkins Build

kibanamachine commented 3 years ago

New failure: Jenkins Build

kibanamachine commented 3 years ago

New failure: Jenkins Build

kibanamachine commented 3 years ago

New failure: Jenkins Build

kibanamachine commented 3 years ago

New failure: Jenkins Build

kibanamachine commented 3 years ago

New failure: Jenkins Build

kibanamachine commented 3 years ago

New failure: Jenkins Build

kibanamachine commented 3 years ago

New failure: Jenkins Build

kibanamachine commented 3 years ago

New failure: Jenkins Build

kibanamachine commented 3 years ago

New failure: Jenkins Build

kibanamachine commented 3 years ago

New failure: Jenkins Build

kibanamachine commented 3 years ago

New failure: Jenkins Build

kibanamachine commented 3 years ago

New failure: Jenkins Build

kibanamachine commented 3 years ago

New failure: Jenkins Build

kibanamachine commented 3 years ago

New failure: Jenkins Build

kibanamachine commented 3 years ago

New failure: Jenkins Build

kibanamachine commented 3 years ago

New failure: Jenkins Build

kibanamachine commented 3 years ago

New failure: Jenkins Build

kibanamachine commented 3 years ago

New failure: Jenkins Build

kibanamachine commented 3 years ago

New failure: Jenkins Build

kibanamachine commented 3 years ago

New failure: Jenkins Build

brianseeders commented 3 years ago

Skipped

master: e9988b1cc2c8f322f63f441891c9032c9b9f25ce 7.x: 5ed51b4e7c2f49f16d462419805993861212e836

kibanamachine commented 3 years ago

New failure: Jenkins Build

kibanamachine commented 3 years ago

New failure: Jenkins Build

kibanamachine commented 3 years ago

New failure: Jenkins Build

kibanamachine commented 3 years ago

New failure: Jenkins Build

elasticmachine commented 3 years ago

Pinging @elastic/kibana-app-services (Team:AppServices)

Dosant commented 3 years ago

A recent change that might have indirectly caused these failures: https://github.com/elastic/kibana/pull/104780

Looks like test failures aren't related to the search sessions, but all the tests just fail to load a flights sample dashboard. They report: "15 of 17 panels loaded".

These two panels in "no results state" are causing the problem, as they don't set a proper of data- attributes to let test suite know that they finished loading.

Dashboard save a search session with relative time Saves and restores a session with relative time ranges

I filed https://github.com/elastic/kibana/issues/106085 and as soon as this is fixed by @elastic/kibana-app , we can re-enable the test.

@wylieconlon, I am also not sure if this is a valid behaviour that those visualizations could produce a "No results" state with sample data. I assume you didn't hit this in the pr https://github.com/elastic/kibana/pull/104780 ? I am not sure if this is some kind of a race condition in lens code or if this is a valid possible behaviour with how sample data works. Do you maybe know? Otherwise, I think it worth creating an issue to investigate and make it clear?

Also a note: today the skipped test is passing, both locally and on ci: https://github.com/elastic/kibana/pull/106071#issuecomment-882486605

wylieconlon commented 3 years ago

@Dosant Yes I think you're right that https://github.com/elastic/kibana/issues/106085 is the cause, but there's something else that's special about those two panels: they are both using the time shift feature. Could that be related, since the data for all the other panels is loaded?

Dosant commented 3 years ago

Could that be related, since the data for all the other panels are loaded?

I am not sure, but an important remark is that the empty state happens not when the test tries to restore a search session, but it happens and fails on the initial dashboard load. So this could likely happen in any other test that would use sample flights data and this empty state is not related to search sessions functionality

kibanamachine commented 1 year ago

New failure: CI Build - main

kibanamachine commented 1 year ago

New failure: CI Build - 8.7

ThomThomson commented 1 year ago

It looks like this is a completely different failure this time:

delayed vs 1 week earlier and canceled vs 1 week earlier both loaded successfully:
The error this time is: The element [data-test-subj="embeddableError"] was still present when it should have disappeared.

My guess is that one of the charts on this dashboard errored out, but it's difficult to tell which one at this point.

I think the next step is to replace instances of await testSubjects.missingOrFail('embeddableError'); with a method on the dashboard service that can actually report which panels have failed, then run a flaky test runner against this.

kibanamachine commented 2 months ago

New failure: kibana-on-merge - main

kibanamachine commented 1 month ago

New failure: kibana-7-dot-17-es-8-dot-13-forward-compatibility - 7.17

elastic / kibana

Failing test: X-Pack Search Sessions Integration.x-pack/test/search_sessions_integration/tests/apps/dashboard/async_search/save_search_session_relative_time·ts - Dashboard save a search sessions with relative time Saves and restores a session with relative time ranges #97701