elastic / kibana

Your window into the Elastic Stack
https://www.elastic.co/products/kibana
Other
19.74k stars 8.14k forks source link

Failing test: X-Pack Reporting API Integration Tests.x-pack/test/reporting_api_integration/reporting_and_security/bwc_generation_urls·ts - Reporting APIs BWC report generation urls Pre 6_2 job posted successfully #93354

Closed kibanamachine closed 2 years ago

kibanamachine commented 3 years ago

A test failed on a tracked branch

Error: expected 500 to equal 200
    at Assertion.assert (/dev/shm/workspace/parallel/12/kibana/packages/kbn-expect/expect.js:100:11)
    at Assertion.be.Assertion.equal (/dev/shm/workspace/parallel/12/kibana/packages/kbn-expect/expect.js:227:8)
    at Assertion.be (/dev/shm/workspace/parallel/12/kibana/packages/kbn-expect/expect.js:69:22)
    at Object.waitForJobToFinish (test/reporting_api_integration/services.ts:76:29)
    at Context.<anonymous> (test/reporting_api_integration/reporting_and_security/bwc_generation_urls.ts:34:9)
    at Object.apply (/dev/shm/workspace/parallel/12/kibana/packages/kbn-test/src/functional_test_runner/lib/mocha/wrap_function.js:73:16)

First failure: Jenkins Build

kibanamachine commented 3 years ago

New failure: Jenkins Build

spalger commented 3 years ago

Most recent failure was caused by Chrome aborting all connections because the network list changed, not something the test can really control.

kibanamachine commented 3 years ago

New failure: Jenkins Build

spalger commented 3 years ago

Same issue, Chrome aborted all requests mid test.

kibanamachine commented 3 years ago

New failure: Jenkins Build

elasticmachine commented 3 years ago

Pinging @elastic/kibana-reporting-services (Team:Reporting Services)

kibanamachine commented 3 years ago

New failure: Jenkins Build

kibanamachine commented 3 years ago

New failure: Jenkins Build

kibanamachine commented 3 years ago

New failure: Jenkins Build

spalger commented 3 years ago

All the recent test failures are timeouts: Timeout of 500000ms exceeded

Looks like they're only happening on 7.x and 7.14 (and one 7.14 backport), seems these tests have only ever existed in 7.x... I think someone from @elastic/kibana-reporting-services should really take a look at this.

image

Skipped

7.x: https://github.com/elastic/kibana/commit/2a74cd9fd0a8249bf587df29259f615e01d58a7e 7.14: https://github.com/elastic/kibana/commit/d25885e544310b03d838eb3cca944bac76fa35b9

kibanamachine commented 3 years ago

New failure: Jenkins Build

tsullivan commented 3 years ago

Test is failing because a queued job is not getting claimed (for some reason). There is only 1 report job, and it stays in pending status always, instead of being updated to processing.

[00:00:14]                 │ proc [kibana]   log   [17:14:49.839] [info][createJob][plugins][reporting] Scheduled PDF reporting task. Task ID: task:782f7e80-f9fe-11eb-88f6-5f719cc763f7. Report ID: ks6bqcra4roo4347b7anozb6
[00:00:14]                 │ debg Waiting for job to finish: /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6
[00:00:16]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:00:17]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:00:19]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:00:20]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:00:22]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:00:23]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:00:25]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:00:26]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:00:28]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:00:29]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:00:31]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:00:32]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:00:34]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:00:35]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:00:37]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:00:38]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:00:40]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:00:41]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:00:43]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:00:44]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:00:46]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:00:47]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:00:48]                 │ proc [kibana]   log   [17:15:23.474] [error][plugins][taskManager] [WorkloadAggregator]: {"error":{"root_cause":[{"type":"illegal_argument_exception","reason":"Text fields are not optimised for operations that require per-document field data like aggregations and sorting, so these operations are disabled by default. Please use a keyword field instead. Alternatively, set fielddata=true on [task.taskType] in order to load field data by uninverting the inverted index. Note that this can use significant memory."}],"type":"search_phase_execution_exception","reason":"all shards failed","phase":"query","grouped":true,"failed_shards":[{"shard":0,"index":".kibana_task_manager_7.15.0_001","node":"Gzv4sGitTU6aUaE8jIy_Ng","reason":{"type":"illegal_argument_exception","reason":"Text fields are not optimised for operations that require per-document field data like aggregations and sorting, so these operations are disabled by default. Please use a keyword field instead. Alternatively, set fielddata=true on [task.taskType] in order to load field data by uninverting the inverted index. Note that this can use significant memory."}}],"caused_by":{"type":"illegal_argument_exception","reason":"Text fields are not optimised for operations that require per-document field data like aggregations and sorting, so these operations are disabled by default. Please use a keyword field instead. Alternatively, set fielddata=true on [task.taskType] in order to load field data by uninverting the inverted index. Note that this can use significant memory.","caused_by":{"type":"illegal_argument_exception","reason":"Text fields are not optimised for operations that require per-document field data like aggregations and sorting, so these operations are disabled by default. Please use a keyword field instead. Alternatively, set fielddata=true on [task.taskType] in order to load field data by uninverting the inverted index. Note that this can use significant memory."}}},"status":400}
[00:00:48]                 │ proc [kibana] {"log.level":"info","@timestamp":"2021-08-10T17:15:23.476Z","log":{"logger":"elastic-apm-node"},"ecs":{"version":"1.6.0"},"message":"Sending error to Elastic APM: {\"id\":\"54b7fc2323219f0cf805b21540dca412\"}"}
[00:00:49]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:00:50]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:00:52]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:00:53]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:00:55]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:00:56]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:00:58]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:00:59]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:01:01]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:01:02]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:01:04]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:01:05]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:01:07]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:01:08]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:01:10]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:01:11]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:01:13]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:01:14]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:01:16]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:01:17]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:01:19]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:01:20]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:01:21]                 │ proc [kibana]   log   [17:15:56.504] [warning][plugins][taskManager] Detected potential performance issue with Task Manager. Set 'xpack.task_manager.monitored_stats_health_verbose_log.enabled: true' in your Kibana.yml to enable debug logging
[00:01:22]                 │ proc [kibana]   log   [17:15:57.257] [info][status] Kibana is now unavailable (was available)
[00:01:22]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:01:23]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:01:25]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:01:27]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:01:28]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:01:29]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:01:31]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:01:32]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:01:34]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:01:36]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:01:37]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:01:39]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:01:40]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:01:42]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:01:43]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:01:45]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:01:46]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:01:48]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:01:48]                 │ proc [kibana]   log   [17:16:23.484] [error][plugins][taskManager] [WorkloadAggregator]: {"error":{"root_cause":[{"type":"illegal_argument_exception","reason":"Text fields are not optimised for operations that require per-document field data like aggregations and sorting, so these operations are disabled by default. Please use a keyword field instead. Alternatively, set fielddata=true on [task.taskType] in order to load field data by uninverting the inverted index. Note that this can use significant memory."}],"type":"search_phase_execution_exception","reason":"all shards failed","phase":"query","grouped":true,"failed_shards":[{"shard":0,"index":".kibana_task_manager_7.15.0_001","node":"Gzv4sGitTU6aUaE8jIy_Ng","reason":{"type":"illegal_argument_exception","reason":"Text fields are not optimised for operations that require per-document field data like aggregations and sorting, so these operations are disabled by default. Please use a keyword field instead. Alternatively, set fielddata=true on [task.taskType] in order to load field data by uninverting the inverted index. Note that this can use significant memory."}}],"caused_by":{"type":"illegal_argument_exception","reason":"Text fields are not optimised for operations that require per-document field data like aggregations and sorting, so these operations are disabled by default. Please use a keyword field instead. Alternatively, set fielddata=true on [task.taskType] in order to load field data by uninverting the inverted index. Note that this can use significant memory.","caused_by":{"type":"illegal_argument_exception","reason":"Text fields are not optimised for operations that require per-document field data like aggregations and sorting, so these operations are disabled by default. Please use a keyword field instead. Alternatively, set fielddata=true on [task.taskType] in order to load field data by uninverting the inverted index. Note that this can use significant memory."}}},"status":400}
[00:01:48]                 │ proc [kibana] {"log.level":"info","@timestamp":"2021-08-10T17:16:23.486Z","log":{"logger":"elastic-apm-node"},"ecs":{"version":"1.6.0"},"message":"Sending error to Elastic APM: {\"id\":\"6fbf45da66b45205a2412ccf9357b08b\"}"}
[00:01:49]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:01:51]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:01:52]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:01:54]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:01:55]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:01:57]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:01:58]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:02:00]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:02:01]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:02:03]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:02:04]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:02:06]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:02:07]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:02:09]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:02:10]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:02:12]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:02:13]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:02:15]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:02:16]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:02:18]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:02:19]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:02:21]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:02:22]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:02:24]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:02:25]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:02:27]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:02:28]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:02:30]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:02:31]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:02:33]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:02:34]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:02:36]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:02:37]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:02:39]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:02:40]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:02:42]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:02:43]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:02:45]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:02:46]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:02:48]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:02:48]                 │ proc [kibana]   log   [17:17:23.493] [error][plugins][taskManager] [WorkloadAggregator]: {"error":{"root_cause":[{"type":"illegal_argument_exception","reason":"Text fields are not optimised for operations that require per-document field data like aggregations and sorting, so these operations are disabled by default. Please use a keyword field instead. Alternatively, set fielddata=true on [task.taskType] in order to load field data by uninverting the inverted index. Note that this can use significant memory."}],"type":"search_phase_execution_exception","reason":"all shards failed","phase":"query","grouped":true,"failed_shards":[{"shard":0,"index":".kibana_task_manager_7.15.0_001","node":"Gzv4sGitTU6aUaE8jIy_Ng","reason":{"type":"illegal_argument_exception","reason":"Text fields are not optimised for operations that require per-document field data like aggregations and sorting, so these operations are disabled by default. Please use a keyword field instead. Alternatively, set fielddata=true on [task.taskType] in order to load field data by uninverting the inverted index. Note that this can use significant memory."}}],"caused_by":{"type":"illegal_argument_exception","reason":"Text fields are not optimised for operations that require per-document field data like aggregations and sorting, so these operations are disabled by default. Please use a keyword field instead. Alternatively, set fielddata=true on [task.taskType] in order to load field data by uninverting the inverted index. Note that this can use significant memory.","caused_by":{"type":"illegal_argument_exception","reason":"Text fields are not optimised for operations that require per-document field data like aggregations and sorting, so these operations are disabled by default. Please use a keyword field instead. Alternatively, set fielddata=true on [task.taskType] in order to load field data by uninverting the inverted index. Note that this can use significant memory."}}},"status":400}
[00:02:48]                 │ proc [kibana] {"log.level":"info","@timestamp":"2021-08-10T17:17:23.496Z","log":{"logger":"elastic-apm-node"},"ecs":{"version":"1.6.0"},"message":"Sending error to Elastic APM: {\"id\":\"deb025d5f714687fc796afcb0f1f90b5\"}"}
[00:02:49]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:02:51]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:02:52]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:02:54]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:02:55]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:02:57]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:02:58]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:03:00]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:03:01]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:03:03]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:03:04]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:03:06]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:03:07]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:03:09]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:03:10]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:03:12]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:03:13]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:03:15]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:03:16]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:03:18]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:03:19]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:03:21]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:03:22]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:03:24]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:03:25]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:03:27]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:03:28]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:03:30]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:03:31]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:03:33]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:03:34]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:03:36]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:03:37]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:03:39]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:03:40]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:03:42]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:03:43]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:03:45]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:03:46]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:03:48]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:03:48]                 │ proc [kibana]   log   [17:18:23.502] [error][plugins][taskManager] [WorkloadAggregator]: {"error":{"root_cause":[{"type":"illegal_argument_exception","reason":"Text fields are not optimised for operations that require per-document field data like aggregations and sorting, so these operations are disabled by default. Please use a keyword field instead. Alternatively, set fielddata=true on [task.taskType] in order to load field data by uninverting the inverted index. Note that this can use significant memory."}],"type":"search_phase_execution_exception","reason":"all shards failed","phase":"query","grouped":true,"failed_shards":[{"shard":0,"index":".kibana_task_manager_7.15.0_001","node":"Gzv4sGitTU6aUaE8jIy_Ng","reason":{"type":"illegal_argument_exception","reason":"Text fields are not optimised for operations that require per-document field data like aggregations and sorting, so these operations are disabled by default. Please use a keyword field instead. Alternatively, set fielddata=true on [task.taskType] in order to load field data by uninverting the inverted index. Note that this can use significant memory."}}],"caused_by":{"type":"illegal_argument_exception","reason":"Text fields are not optimised for operations that require per-document field data like aggregations and sorting, so these operations are disabled by default. Please use a keyword field instead. Alternatively, set fielddata=true on [task.taskType] in order to load field data by uninverting the inverted index. Note that this can use significant memory.","caused_by":{"type":"illegal_argument_exception","reason":"Text fields are not optimised for operations that require per-document field data like aggregations and sorting, so these operations are disabled by default. Please use a keyword field instead. Alternatively, set fielddata=true on [task.taskType] in order to load field data by uninverting the inverted index. Note that this can use significant memory."}}},"status":400}
[00:03:48]                 │ proc [kibana] {"log.level":"info","@timestamp":"2021-08-10T17:18:23.504Z","log":{"logger":"elastic-apm-node"},"ecs":{"version":"1.6.0"},"message":"Sending error to Elastic APM: {\"id\":\"f6790253a709094555c4a33dc282264f\"}"}
[00:03:49]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:03:51]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:03:52]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:03:54]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:03:55]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:03:57]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:03:58]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:04:00]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:04:01]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:04:03]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:04:04]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:04:06]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:04:07]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:04:09]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:04:10]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:04:12]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:04:13]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:04:15]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:04:16]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:04:18]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:04:19]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:04:21]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:04:22]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:04:24]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:04:25]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:04:27]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:04:28]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:04:30]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:04:31]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:04:33]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:04:34]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:04:36]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:04:37]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:04:39]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:04:40]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:04:42]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:04:43]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:04:45]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:04:46]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:04:48]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:04:48]                 │ proc [kibana]   log   [17:19:23.511] [error][plugins][taskManager] [WorkloadAggregator]: {"error":{"root_cause":[{"type":"illegal_argument_exception","reason":"Text fields are not optimised for operations that require per-document field data like aggregations and sorting, so these operations are disabled by default. Please use a keyword field instead. Alternatively, set fielddata=true on [task.taskType] in order to load field data by uninverting the inverted index. Note that this can use significant memory."}],"type":"search_phase_execution_exception","reason":"all shards failed","phase":"query","grouped":true,"failed_shards":[{"shard":0,"index":".kibana_task_manager_7.15.0_001","node":"Gzv4sGitTU6aUaE8jIy_Ng","reason":{"type":"illegal_argument_exception","reason":"Text fields are not optimised for operations that require per-document field data like aggregations and sorting, so these operations are disabled by default. Please use a keyword field instead. Alternatively, set fielddata=true on [task.taskType] in order to load field data by uninverting the inverted index. Note that this can use significant memory."}}],"caused_by":{"type":"illegal_argument_exception","reason":"Text fields are not optimised for operations that require per-document field data like aggregations and sorting, so these operations are disabled by default. Please use a keyword field instead. Alternatively, set fielddata=true on [task.taskType] in order to load field data by uninverting the inverted index. Note that this can use significant memory.","caused_by":{"type":"illegal_argument_exception","reason":"Text fields are not optimised for operations that require per-document field data like aggregations and sorting, so these operations are disabled by default. Please use a keyword field instead. Alternatively, set fielddata=true on [task.taskType] in order to load field data by uninverting the inverted index. Note that this can use significant memory."}}},"status":400}
[00:04:48]                 │ proc [kibana] {"log.level":"info","@timestamp":"2021-08-10T17:19:23.516Z","log":{"logger":"elastic-apm-node"},"ecs":{"version":"1.6.0"},"message":"Sending error to Elastic APM: {\"id\":\"1259a182fbe6c519f187176586a83dcc\"}"}
[00:04:49]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:04:49]                 │ info [o.e.c.m.MetadataMappingService] [node-01] [.kibana_7.15.0_001/sDFjNCyDQ9-0S4bwVc2xtQ] update_mapping [_doc]
[00:04:50]                 │ info [o.e.c.m.MetadataMappingService] [node-01] [.kibana_7.15.0_001/sDFjNCyDQ9-0S4bwVc2xtQ] update_mapping [_doc]
[00:04:50]                 │ info [o.e.x.i.IndexLifecycleTransition] [node-01] moving index [.reporting-2021-08-08] from [{"phase":"new","action":"complete","name":"complete"}] to [{"phase":"hot","action":"complete","name":"complete"}] in policy [kibana-reporting]
[00:04:50]                 │ proc [kibana]   log   [17:19:25.135] [warning][collector-set][plugins][usage-collection][usageCollection] ReferenceError: Cannot access 'error' before initialization
[00:04:50]                 │ proc [kibana]     at /dev/shm/workspace/kibana-build-7/x-pack/plugins/fleet/server/collectors/fleet_server_collector.js:43:27
[00:04:50]                 │ proc [kibana]     at runMicrotasks (<anonymous>)
[00:04:50]                 │ proc [kibana]     at processTicksAndRejections (internal/process/task_queues.js:95:5)
[00:04:50]                 │ proc [kibana]     at getFleetServerUsage (/dev/shm/workspace/kibana-build-7/x-pack/plugins/fleet/server/collectors/fleet_server_collector.js:38:24)
[00:04:50]                 │ proc [kibana]     at UsageCollector.fetch (/dev/shm/workspace/kibana-build-7/x-pack/plugins/fleet/server/collectors/register.js:41:23)
[00:04:50]                 │ proc [kibana]     at /dev/shm/workspace/kibana-build-7/src/plugins/usage_collection/server/collector/collector_set.js:104:21
[00:04:50]                 │ proc [kibana]     at async Promise.all (index 37)
[00:04:50]                 │ proc [kibana]     at Object.bulkFetch (/dev/shm/workspace/kibana-build-7/src/plugins/usage_collection/server/collector/collector_set.js:91:25)
[00:04:50]                 │ proc [kibana]     at getKibana (/dev/shm/workspace/kibana-build-7/src/plugins/telemetry/server/telemetry_collection/get_kibana.js:68:17)
[00:04:50]                 │ proc [kibana]     at async Promise.all (index 3)
[00:04:50]                 │ proc [kibana]     at /dev/shm/workspace/kibana-build-7/src/plugins/telemetry/server/telemetry_collection/get_local_stats.js:79:76
[00:04:50]                 │ proc [kibana]     at async Promise.all (index 0)
[00:04:50]                 │ proc [kibana]     at getLocalStats (/dev/shm/workspace/kibana-build-7/src/plugins/telemetry/server/telemetry_collection/get_local_stats.js:78:10)
[00:04:50]                 │ proc [kibana]     at async Promise.all (index 0)
[00:04:50]                 │ proc [kibana]     at Object.getStatsWithXpack [as statsGetter] (/dev/shm/workspace/kibana-build-7/x-pack/plugins/telemetry_collection_xpack/server/telemetry_collection/get_stats_with_xpack.js:27:48)
[00:04:50]                 │ proc [kibana]     at TelemetryCollectionManagerPlugin.getUsageForCollection (/dev/shm/workspace/kibana-build-7/src/plugins/telemetry_collection_manager/server/plugin.js:256:19)
[00:04:50]                 │ proc [kibana]     at TelemetryCollectionManagerPlugin.getStats (/dev/shm/workspace/kibana-build-7/src/plugins/telemetry_collection_manager/server/plugin.js:222:29)
[00:04:50]                 │ proc [kibana]     at FetcherTask.fetchTelemetry (/dev/shm/workspace/kibana-build-7/src/plugins/telemetry/server/fetcher.js:215:12)
[00:04:50]                 │ proc [kibana]     at FetcherTask.sendIfDue (/dev/shm/workspace/kibana-build-7/src/plugins/telemetry/server/fetcher.js:111:18)
[00:04:50]                 │ proc [kibana]   log   [17:19:25.139] [warning][collector-set][plugins][usage-collection][usageCollection] Unable to fetch data from fleet collector
[00:04:50]                 │ proc [kibana] {"log.level":"info","@timestamp":"2021-08-10T17:19:25.419Z","log":{"logger":"elastic-apm-node"},"ecs":{"version":"1.6.0"},"message":"Sending error to Elastic APM: {\"id\":\"92f27ef6be752fe495b80287273b4dfb\"}"}
[00:04:51]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:04:51]                 │ proc [kibana] {"log.level":"info","@timestamp":"2021-08-10T17:19:26.234Z","log":{"logger":"elastic-apm-node"},"ecs":{"version":"1.6.0"},"message":"Sending error to Elastic APM: {\"id\":\"c9404d284dba4525c66385b20aed37ae\"}"}
[00:04:52]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:04:54]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:04:55]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:04:57]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:04:58]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:05:00]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:05:01]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:05:03]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:05:04]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:05:06]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:05:07]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:05:09]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:05:10]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:05:12]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:05:13]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:05:15]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:05:16]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:05:18]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:05:19]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:05:21]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:05:22]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:05:24]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:05:25]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:05:27]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:05:28]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:05:30]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:05:31]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:05:33]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:05:34]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:05:36]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:05:37]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:05:39]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:05:40]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:05:42]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:05:43]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:05:45]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:05:46]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:05:48]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:05:48]                 │ proc [kibana]   log   [17:20:23.518] [error][plugins][taskManager] [WorkloadAggregator]: {"error":{"root_cause":[{"type":"illegal_argument_exception","reason":"Text fields are not optimised for operations that require per-document field data like aggregations and sorting, so these operations are disabled by default. Please use a keyword field instead. Alternatively, set fielddata=true on [task.taskType] in order to load field data by uninverting the inverted index. Note that this can use significant memory."}],"type":"search_phase_execution_exception","reason":"all shards failed","phase":"query","grouped":true,"failed_shards":[{"shard":0,"index":".kibana_task_manager_7.15.0_001","node":"Gzv4sGitTU6aUaE8jIy_Ng","reason":{"type":"illegal_argument_exception","reason":"Text fields are not optimised for operations that require per-document field data like aggregations and sorting, so these operations are disabled by default. Please use a keyword field instead. Alternatively, set fielddata=true on [task.taskType] in order to load field data by uninverting the inverted index. Note that this can use significant memory."}}],"caused_by":{"type":"illegal_argument_exception","reason":"Text fields are not optimised for operations that require per-document field data like aggregations and sorting, so these operations are disabled by default. Please use a keyword field instead. Alternatively, set fielddata=true on [task.taskType] in order to load field data by uninverting the inverted index. Note that this can use significant memory.","caused_by":{"type":"illegal_argument_exception","reason":"Text fields are not optimised for operations that require per-document field data like aggregations and sorting, so these operations are disabled by default. Please use a keyword field instead. Alternatively, set fielddata=true on [task.taskType] in order to load field data by uninverting the inverted index. Note that this can use significant memory."}}},"status":400}
[00:05:48]                 │ proc [kibana] {"log.level":"info","@timestamp":"2021-08-10T17:20:23.520Z","log":{"logger":"elastic-apm-node"},"ecs":{"version":"1.6.0"},"message":"Sending error to Elastic APM: {\"id\":\"badd9967477dcc2010c4f89ef11726d6\"}"}
[00:05:49]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:05:51]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:05:52]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:05:54]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:05:55]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:05:57]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:05:58]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:06:00]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:06:01]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:06:03]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:06:04]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:06:06]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:06:07]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:06:09]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:06:10]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:06:12]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:06:13]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:06:15]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:06:16]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:06:18]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:06:19]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:06:21]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:06:22]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:06:24]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:06:25]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:06:27]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:06:28]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:06:30]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:06:31]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:06:33]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:06:34]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:06:36]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:06:37]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:06:39]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:06:40]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:06:42]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:06:43]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:06:45]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:06:46]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:06:48]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:06:48]                 │ proc [kibana]   log   [17:21:23.528] [error][plugins][taskManager] [WorkloadAggregator]: {"error":{"root_cause":[{"type":"illegal_argument_exception","reason":"Text fields are not optimised for operations that require per-document field data like aggregations and sorting, so these operations are disabled by default. Please use a keyword field instead. Alternatively, set fielddata=true on [task.taskType] in order to load field data by uninverting the inverted index. Note that this can use significant memory."}],"type":"search_phase_execution_exception","reason":"all shards failed","phase":"query","grouped":true,"failed_shards":[{"shard":0,"index":".kibana_task_manager_7.15.0_001","node":"Gzv4sGitTU6aUaE8jIy_Ng","reason":{"type":"illegal_argument_exception","reason":"Text fields are not optimised for operations that require per-document field data like aggregations and sorting, so these operations are disabled by default. Please use a keyword field instead. Alternatively, set fielddata=true on [task.taskType] in order to load field data by uninverting the inverted index. Note that this can use significant memory."}}],"caused_by":{"type":"illegal_argument_exception","reason":"Text fields are not optimised for operations that require per-document field data like aggregations and sorting, so these operations are disabled by default. Please use a keyword field instead. Alternatively, set fielddata=true on [task.taskType] in order to load field data by uninverting the inverted index. Note that this can use significant memory.","caused_by":{"type":"illegal_argument_exception","reason":"Text fields are not optimised for operations that require per-document field data like aggregations and sorting, so these operations are disabled by default. Please use a keyword field instead. Alternatively, set fielddata=true on [task.taskType] in order to load field data by uninverting the inverted index. Note that this can use significant memory."}}},"status":400}
[00:06:48]                 │ proc [kibana] {"log.level":"info","@timestamp":"2021-08-10T17:21:23.531Z","log":{"logger":"elastic-apm-node"},"ecs":{"version":"1.6.0"},"message":"Sending error to Elastic APM: {\"id\":\"e1bf4ba562d0921be6ecf135d563b8c3\"}"}
[00:06:49]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:06:51]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:06:52]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:06:54]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:06:55]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:06:57]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:06:58]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:07:00]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:07:01]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:07:03]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:07:04]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:07:06]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:07:07]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:07:09]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:07:10]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:07:12]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:07:13]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:07:15]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:07:16]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:07:18]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:07:19]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:07:21]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:07:22]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:07:24]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:07:25]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:07:27]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:07:28]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:07:30]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:07:31]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:07:33]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:07:34]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:07:36]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:07:37]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:07:39]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:07:40]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:07:42]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:07:43]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:07:45]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:07:46]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:07:48]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:07:48]                 │ proc [kibana]   log   [17:22:23.536] [error][plugins][taskManager] [WorkloadAggregator]: {"error":{"root_cause":[{"type":"illegal_argument_exception","reason":"Text fields are not optimised for operations that require per-document field data like aggregations and sorting, so these operations are disabled by default. Please use a keyword field instead. Alternatively, set fielddata=true on [task.taskType] in order to load field data by uninverting the inverted index. Note that this can use significant memory."}],"type":"search_phase_execution_exception","reason":"all shards failed","phase":"query","grouped":true,"failed_shards":[{"shard":0,"index":".kibana_task_manager_7.15.0_001","node":"Gzv4sGitTU6aUaE8jIy_Ng","reason":{"type":"illegal_argument_exception","reason":"Text fields are not optimised for operations that require per-document field data like aggregations and sorting, so these operations are disabled by default. Please use a keyword field instead. Alternatively, set fielddata=true on [task.taskType] in order to load field data by uninverting the inverted index. Note that this can use significant memory."}}],"caused_by":{"type":"illegal_argument_exception","reason":"Text fields are not optimised for operations that require per-document field data like aggregations and sorting, so these operations are disabled by default. Please use a keyword field instead. Alternatively, set fielddata=true on [task.taskType] in order to load field data by uninverting the inverted index. Note that this can use significant memory.","caused_by":{"type":"illegal_argument_exception","reason":"Text fields are not optimised for operations that require per-document field data like aggregations and sorting, so these operations are disabled by default. Please use a keyword field instead. Alternatively, set fielddata=true on [task.taskType] in order to load field data by uninverting the inverted index. Note that this can use significant memory."}}},"status":400}
[00:07:48]                 │ proc [kibana] {"log.level":"info","@timestamp":"2021-08-10T17:22:23.538Z","log":{"logger":"elastic-apm-node"},"ecs":{"version":"1.6.0"},"message":"Sending error to Elastic APM: {\"id\":\"7534f4f8885633114e914a756bfd1868\"}"}
[00:07:49]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:07:51]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:07:52]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:07:54]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:07:55]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:07:57]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:07:58]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:08:00]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:08:01]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:08:03]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:08:04]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:08:06]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:08:07]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:08:09]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:08:10]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:08:12]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:08:13]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:08:15]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:08:16]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:08:18]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:08:19]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:08:21]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:08:22]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:08:24]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:08:25]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:08:27]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:08:28]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:08:30]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:08:31]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:08:33]                 │ debg Report at path /api/reporting/jobs/download/ks6bqcra4roo4347b7anozb6 is pending
[00:08:34]                 └- ✖ fail: Reporting APIs BWC report generation urls Pre 6_2 job posted successfully
[00:08:34]                 │      Error: Timeout of 500000ms exceeded. For async tests and hooks, ensure "done()" is called; if returning a Promise, ensure it resolves. (/dev/shm/workspace/parallel/7/kibana/x-pack/test/reporting_api_integration/reporting_and_security/bwc_generation_urls.ts)
[00:08:34]                 │       at listOnTimeout (internal/timers.js:557:17)
[00:08:34]                 │       at processTimers (internal/timers.js:500:7)
kibanamachine commented 3 years ago

New failure: Jenkins Build

kibanamachine commented 2 years ago

New failure: CI Build

mistic commented 2 years ago

Looks like this test started to failing again. Skipping for now.

master: 5f26444 7.x/7.16: https://github.com/elastic/kibana/commit/fc88c9dfa0142af9d94ad8b0e31a141de09a88b3 7.15: https://github.com/elastic/kibana/commit/c7532e9ba8abd3eabc711830f723369b140d06d1

kibanamachine commented 2 years ago

New failure: CI Build

kibanamachine commented 2 years ago

New failure: CI Build - 7.x

spalger commented 2 years ago

Skipped in 7.x and 7.15 as well

tsullivan commented 2 years ago

This test was hitting issues with Task Manager. I'm not sure, but that could be due to the way the test used es_archiver to load Kibana objects. Those types of problems are resolved in https://github.com/elastic/kibana/pull/116528

tsullivan commented 2 years ago

Replacing with: https://github.com/elastic/kibana/issues/120192