Fix checksum validation - Githubissues

elastic / beats-tester

E2E tests for the Beats

18 stars 22 forks source link

Fix checksum validation #201

Closed adam-stokes closed 3 years ago

adam-stokes commented 3 years ago

Attempt to fix issue where get_url doesn't properly read a checksum value out of a checksum file:

https://github.com/ansible/ansible/issues/54390

Fixes https://github.com/elastic/observability-robots/issues/579

elasticmachine commented 3 years ago

:broken_heart: Tests Failed

the below badges are clickable and redirect to their specific view in the CI or DOCS

Expand to view the summary

#### Build stats * Start Time: 2021-07-26T17:13:44.218+0000 * Duration: 18 min 23 sec * Commit: 1b8330120dab69cba0c2449658399d9ee8d8a019 #### Test stats :test_tube: | Test | Results | | ------------ | :-----------------------------: | | Failed | 12 | | Passed | 2929 | | Skipped | 1813 | | Total | 4754 | #### Trends :test_tube: ![Image of Build Times](https://beats-ci.elastic.co/job/Beats/job/beats-tester-mbp/job/PR-201/buildTimeGraph/png) ![Image of Tests](https://beats-ci.elastic.co/job/Beats/job/beats-tester-mbp/job/PR-201/13/testReport/history/countGraph/png?start=0&end=25)

Test errors

Expand to view the tests failures

> Show only the first 10 test failures ##### `Test Hosts / Matrix - GROUPS = 'centos' / Test / [tester-awslinux] Packaging tests Heartbeat: test-linux-binary : Wait for the output file to be created, should contain HTTP pinging search_regex="scheme":"http", path={{workdir}}/output/heartbeat, timeout=5 – /var/lib/jenkins/workspace/Beats_beats-tester-mbp_PR-201/src/roles/test-linux-binary/tasks/main.yml:74`

Expand to view the error details

``` Timeout when waiting for search string "scheme":"http" in /tmp/output/heartbeat ```

Expand to view the stacktrace

``` { "changed": false, "elapsed": 5, "msg": "Timeout when waiting for search string \"scheme\":\"http\" in /tmp/output/heartbeat" } ```

##### `Test Hosts / Matrix - GROUPS = 'centos' / Test / [tester-centos7-64] Packaging tests Heartbeat: test-linux-binary : Wait for the output file to be created, should contain HTTP pinging search_regex="scheme":"http", path={{workdir}}/output/heartbeat, timeout=5 – /var/lib/jenkins/workspace/Beats_beats-tester-mbp_PR-201/src/roles/test-linux-binary/tasks/main.yml:74`

Expand to view the error details

``` Timeout when waiting for search string "scheme":"http" in /tmp/output/heartbeat ```

Expand to view the stacktrace

``` { "changed": false, "elapsed": 5, "msg": "Timeout when waiting for search string \"scheme\":\"http\" in /tmp/output/heartbeat" } ```

##### `Test Hosts / Matrix - GROUPS = 'centos' / Test / [tester-awslinux2] Packaging tests Heartbeat: test-linux-binary : Wait for the output file to be created, should contain HTTP pinging search_regex="scheme":"http", path={{workdir}}/output/heartbeat, timeout=5 – /var/lib/jenkins/workspace/Beats_beats-tester-mbp_PR-201/src/roles/test-linux-binary/tasks/main.yml:74`

Expand to view the error details

``` Timeout when waiting for search string "scheme":"http" in /tmp/output/heartbeat ```

Expand to view the stacktrace

``` { "changed": false, "elapsed": 5, "msg": "Timeout when waiting for search string \"scheme\":\"http\" in /tmp/output/heartbeat" } ```

##### `Test Hosts / Matrix - GROUPS = 'centos' / Test / [tester-centos8-64] Packaging tests Heartbeat: test-linux-binary : Wait for the output file to be created, should contain HTTP pinging search_regex="scheme":"http", path={{workdir}}/output/heartbeat, timeout=5 – /var/lib/jenkins/workspace/Beats_beats-tester-mbp_PR-201/src/roles/test-linux-binary/tasks/main.yml:74`

Expand to view the error details

``` Timeout when waiting for search string "scheme":"http" in /tmp/output/heartbeat ```

Expand to view the stacktrace

``` { "changed": false, "elapsed": 5, "msg": "Timeout when waiting for search string \"scheme\":\"http\" in /tmp/output/heartbeat" } ```

##### `Test Hosts / Matrix - GROUPS = 'sles' / Test / [tester-sles12-64] Packaging tests Heartbeat: test-linux-binary : Wait for the output file to be created, should contain HTTP pinging search_regex="scheme":"http", path={{workdir}}/output/heartbeat, timeout=5 – /var/lib/jenkins/workspace/Beats_beats-tester-mbp_PR-201/src/roles/test-linux-binary/tasks/main.yml:74`

Expand to view the error details

``` Timeout when waiting for search string "scheme":"http" in /tmp/output/heartbeat ```

Expand to view the stacktrace

``` { "changed": false, "elapsed": 5, "msg": "Timeout when waiting for search string \"scheme\":\"http\" in /tmp/output/heartbeat" } ```

##### `Test Hosts / Matrix - GROUPS = 'debian' / Test / [tester-debian9-64] Packaging tests Heartbeat: test-linux-binary : Wait for the output file to be created, should contain HTTP pinging search_regex="scheme":"http", path={{workdir}}/output/heartbeat, timeout=5 – /var/lib/jenkins/workspace/Beats_beats-tester-mbp_PR-201/src/roles/test-linux-binary/tasks/main.yml:74`

Expand to view the error details

``` Timeout when waiting for search string "scheme":"http" in /tmp/output/heartbeat ```

Expand to view the stacktrace

``` { "changed": false, "elapsed": 5, "msg": "Timeout when waiting for search string \"scheme\":\"http\" in /tmp/output/heartbeat" } ```

##### `Test Hosts / Matrix - GROUPS = 'debian' / Test / [tester-debian10-64] Packaging tests Heartbeat: test-linux-binary : Wait for the output file to be created, should contain HTTP pinging search_regex="scheme":"http", path={{workdir}}/output/heartbeat, timeout=5 – /var/lib/jenkins/workspace/Beats_beats-tester-mbp_PR-201/src/roles/test-linux-binary/tasks/main.yml:74`

Expand to view the error details

``` Timeout when waiting for search string "scheme":"http" in /tmp/output/heartbeat ```

Expand to view the stacktrace

``` { "changed": false, "elapsed": 5, "msg": "Timeout when waiting for search string \"scheme\":\"http\" in /tmp/output/heartbeat" } ```

##### `Test Hosts / Matrix - GROUPS = 'debian' / Test / [tester-ubuntu1804-64] Packaging tests Heartbeat: test-linux-binary : Wait for the output file to be created, should contain HTTP pinging search_regex="scheme":"http", path={{workdir}}/output/heartbeat, timeout=5 – /var/lib/jenkins/workspace/Beats_beats-tester-mbp_PR-201/src/roles/test-linux-binary/tasks/main.yml:74`

Expand to view the error details

``` Timeout when waiting for search string "scheme":"http" in /tmp/output/heartbeat ```

Expand to view the stacktrace

``` { "changed": false, "elapsed": 5, "msg": "Timeout when waiting for search string \"scheme\":\"http\" in /tmp/output/heartbeat" } ```

##### `Test Hosts / Matrix - GROUPS = 'debian' / Test / [tester-debian8-64] Packaging tests Heartbeat: test-linux-binary : Wait for the output file to be created, should contain HTTP pinging search_regex="scheme":"http", path={{workdir}}/output/heartbeat, timeout=5 – /var/lib/jenkins/workspace/Beats_beats-tester-mbp_PR-201/src/roles/test-linux-binary/tasks/main.yml:74`

Expand to view the error details

``` Timeout when waiting for search string "scheme":"http" in /tmp/output/heartbeat ```

Expand to view the stacktrace

``` { "changed": false, "elapsed": 5, "msg": "Timeout when waiting for search string \"scheme\":\"http\" in /tmp/output/heartbeat" } ```

##### `Test Hosts / Matrix - GROUPS = 'debian' / Test / [tester-ubuntu2004-64] Packaging tests Heartbeat: test-linux-binary : Wait for the output file to be created, should contain HTTP pinging search_regex="scheme":"http", path={{workdir}}/output/heartbeat, timeout=5 – /var/lib/jenkins/workspace/Beats_beats-tester-mbp_PR-201/src/roles/test-linux-binary/tasks/main.yml:74`

Expand to view the error details

``` Timeout when waiting for search string "scheme":"http" in /tmp/output/heartbeat ```

Expand to view the stacktrace

``` { "changed": false, "elapsed": 5, "msg": "Timeout when waiting for search string \"scheme\":\"http\" in /tmp/output/heartbeat" } ```

Steps errors

Expand to view the steps failures

##### `make batch`

Took 7 min 24 sec . View more details on here

##### `make batch`

Took 13 min 13 sec . View more details on here

##### `make batch`

Took 5 min 39 sec . View more details on here

##### `make batch`

Took 2 min 4 sec . View more details on here

##### `Recursively delete the current directory from the workspace`

Took 0 min 1 sec . View more details on here
Description: script returned exit code 2

Log output

Expand to view the last 100 lines of log output

``` [2021-07-26T17:31:29.620Z] changed: [tester-win12-64] [2021-07-26T17:31:29.620Z] [2021-07-26T17:31:29.620Z] TASK [test-beat : Replace configuration file with template (windows)] ********** [2021-07-26T17:31:32.747Z] changed: [tester-win12-64] [2021-07-26T17:31:32.747Z] [2021-07-26T17:31:32.747Z] TASK [test-beat : Start apm-server Windows service] **************************** [2021-07-26T17:31:34.994Z] fatal: [tester-win12-64]: FAILED! => {"can_pause_and_continue": false, "changed": false, "depended_by": [], "dependencies": [], "description": "", "desktop_interact": false, "display_name": "apm-server", "exists": true, "msg": "Service 'apm-server (apm-server)' cannot be started due to the following error: Cannot start service apm-server on computer '.'.", "name": "apm-server", "path": "\"C:\\Users\\Administrator\\apm-server-8.0.0-SNAPSHOT-windows-x86\\apm-server.exe\" --environment=windows_service -c \"C:\\Users\\Administrator\\apm-server-8.0.0-SNAPSHOT-windows-x86\\apm-server.yml\" --path.home \"C:\\Users\\Administrator\\apm-server-8.0.0-SNAPSHOT-windows-x86\" --path.data \"C:\\ProgramData\\apm-server\" --path.logs \"C:\\ProgramData\\apm-server\\logs\" -E logging.files.redirect_stderr=true", "start_mode": "delayed", "state": "stopped", "username": "LocalSystem"} [2021-07-26T17:31:34.994Z] [2021-07-26T17:31:34.994Z] TASK [test-beat : Save output to host] ***************************************** [2021-07-26T17:31:37.242Z] fatal: [tester-win12-64]: FAILED! => {"changed": false, "msg": "Path c:\\ProgramData\\apm-server\\logs\\output.json is not found"} [2021-07-26T17:31:37.242Z] [2021-07-26T17:31:37.242Z] PLAY RECAP ********************************************************************* [2021-07-26T17:31:37.242Z] tester-win12-64 : ok=18 changed=5 unreachable=0 failed=2 skipped=18 rescued=0 ignored=0 [2021-07-26T17:31:37.242Z] [2021-07-26T17:31:37.242Z] Makefile:51: recipe for target 'ansible-playbook-run-elastic' failed [2021-07-26T17:31:37.242Z] make[2]: *** [ansible-playbook-run-elastic] Error 2 [2021-07-26T17:31:37.242Z] make[2]: Leaving directory '/var/lib/jenkins/workspace/Beats_beats-tester-mbp_PR-201/src' [2021-07-26T17:31:37.242Z] Makefile:36: recipe for target 'run-group' failed [2021-07-26T17:31:37.242Z] make[1]: *** [run-group] Error 2 [2021-07-26T17:31:37.242Z] make[1]: Leaving directory '/var/lib/jenkins/workspace/Beats_beats-tester-mbp_PR-201/src' [2021-07-26T17:31:37.242Z] Makefile:29: recipe for target 'batch' failed [2021-07-26T17:31:37.242Z] make: *** [batch] Error 1 [2021-07-26T17:31:37.595Z] Post stage [2021-07-26T17:31:37.608Z] Running in /var/lib/jenkins/workspace/Beats_beats-tester-mbp_PR-201/src [2021-07-26T17:31:37.730Z] Recording test results [2021-07-26T17:31:38.457Z] [Checks API] No suitable checks publisher found. [2021-07-26T17:31:38.580Z] Archiving artifacts [2021-07-26T17:31:40.072Z] + make clean [2021-07-26T17:31:40.073Z] vagrant destroy -f [2021-07-26T17:31:46.416Z] ==> tester-ubuntu2004-64: VM not created. Moving on... [2021-07-26T17:31:46.416Z] ==> tester-debian10-64: VM not created. Moving on... [2021-07-26T17:31:46.416Z] ==> tester-win19-64: VM not created. Moving on... [2021-07-26T17:31:46.416Z] ==> tester-centos8-64: VM not created. Moving on... [2021-07-26T17:31:46.416Z] ==> tester-awslinux2: VM not created. Moving on... [2021-07-26T17:31:46.417Z] ==> tester-ubuntu1804-64: VM not created. Moving on... [2021-07-26T17:31:46.417Z] ==> tester-sles12-64: VM not created. Moving on... [2021-07-26T17:31:47.112Z] ==> tester-debian9-64: VM not created. Moving on... [2021-07-26T17:31:47.112Z] ==> tester-awslinux: VM not created. Moving on... [2021-07-26T17:31:47.809Z] ==> tester-win12-64: Forcing shutdown of VM... [2021-07-26T17:31:50.060Z] ==> tester-win12-64: Destroying VM and associated drives... [2021-07-26T17:31:51.506Z] ==> tester-centos7-64: VM not created. Moving on... [2021-07-26T17:31:51.506Z] ==> tester-debian8-64: VM not created. Moving on... [2021-07-26T17:31:51.506Z] Makefile:71: recipe for target 'clean' failed [2021-07-26T17:31:51.506Z] make: [clean] Error 2 (ignored) [2021-07-26T17:31:51.506Z] rm -r .vagrant [2021-07-26T17:31:52.565Z] Failed in branch Matrix - GROUPS = 'windows' [2021-07-26T17:31:52.907Z] Running on Jenkins in /var/lib/jenkins/workspace/Beats_beats-tester-mbp_PR-201 [2021-07-26T17:31:53.141Z] [INFO] getVaultSecret: Getting secrets [2021-07-26T17:31:53.173Z] Masking supported pattern matches of $VAULT_ADDR or $VAULT_ROLE_ID or $VAULT_SECRET_ID [2021-07-26T17:31:53.811Z] + chmod 755 generate-build-data.sh [2021-07-26T17:31:53.811Z] + ./generate-build-data.sh https://beats-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/Beats/beats-tester-mbp/PR-201/ https://beats-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/Beats/beats-tester-mbp/PR-201/runs/13 FAILURE 1089332 [2021-07-26T17:31:53.812Z] INFO: curl https://beats-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/Beats/beats-tester-mbp/PR-201/runs/13/steps/?limit=10000 -o steps-info.json [2021-07-26T17:31:55.155Z] INFO: curl https://beats-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/Beats/beats-tester-mbp/PR-201/runs/13/tests/?status=FAILED -o tests-errors.json [2021-07-26T17:31:55.405Z] INFO: curl https://beats-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/Beats/beats-tester-mbp/PR-201/runs/13/log/ -o pipeline-log.txt [2021-07-26T17:31:55.405Z] INFO: curl https://beats-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/Beats/beats-tester-mbp/PR-201// -o job-info.json [2021-07-26T17:31:55.655Z] jq: error (at job-info.json:0): Cannot index string with string "_links" [2021-07-26T17:31:55.656Z] jq: error (at job-info.json:0): Cannot index string with string "_class" [2021-07-26T17:31:55.656Z] jq: error (at job-info.json:0): Cannot index string with string "downloadable" [2021-07-26T17:31:55.656Z] jq: error (at job-info.json:0): Cannot index string with string "id" [2021-07-26T17:31:55.656Z] jq: error (at job-info.json:0): Cannot index string with string "url" [2021-07-26T17:31:55.656Z] jq: error (at job-info.json:21): Cannot index boolean with string "author" [2021-07-26T17:31:55.656Z] jq: error (at job-info.json:21): Cannot index boolean with string "author" [2021-07-26T17:31:55.656Z] INFO: curl https://beats-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/Beats/beats-tester-mbp/PR-201/runs/13/changeSet/ -o changeSet-info.json [2021-07-26T17:31:55.656Z] jq: error (at changeSet-info.json:30): Cannot index array with string "_links" [2021-07-26T17:31:55.906Z] jq: error (at changeSet-info.json:30): Cannot index array with string "_class" [2021-07-26T17:31:55.906Z] jq: error (at changeSet-info.json:30): Cannot index array with string "actions" [2021-07-26T17:31:55.906Z] jq: error (at changeSet-info.json:30): Cannot index array with string "latestRun" [2021-07-26T17:31:55.906Z] jq: error (at changeSet-info.json:30): Cannot index array with string "permissions" [2021-07-26T17:31:55.906Z] jq: error (at changeSet-info.json:30): Cannot index array with string "parameters" [2021-07-26T17:31:55.906Z] INFO: fetchAndPrepareArtifactsInfo (see artifacts-info.json) [2021-07-26T17:31:55.906Z] INFO: curl https://beats-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/Beats/beats-tester-mbp/PR-201/runs/13/artifacts/ -o artifacts-info.json [2021-07-26T17:31:55.906Z] INFO: fetchAndPrepareTestsInfo (see tests-info.json) [2021-07-26T17:31:55.906Z] INFO: curl https://beats-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/Beats/beats-tester-mbp/PR-201/runs/13/tests/?limit=10000000 -o tests-info.json [2021-07-26T17:32:00.028Z] INFO: fetchAndPrepareTestSummaryReport (see tests-summary.json) [2021-07-26T17:32:00.028Z] INFO: curl https://beats-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/Beats/beats-tester-mbp/PR-201/runs/13/blueTestSummary/ -o tests-summary.json [2021-07-26T17:32:00.028Z] INFO: fetchAndPrepareBuildInfo (see build-info.json) [2021-07-26T17:32:00.028Z] INFO: curl https://beats-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/Beats/beats-tester-mbp/PR-201/runs/13/ -o build-info.json [2021-07-26T17:32:00.590Z] Archiving artifacts [2021-07-26T17:32:00.622Z] Timeout set to expire in 5 min 0 sec [2021-07-26T17:32:00.908Z] Archiving artifacts [2021-07-26T17:32:01.140Z] [WARN] notifyIfPossible: could not fetch the nextBuild. [2021-07-26T17:32:01.324Z] Copied 1 artifact from "Beats » Pipeline for beats-tester » PR-201" build number 12 [2021-07-26T17:32:02.803Z] Archiving artifacts [2021-07-26T17:32:02.873Z] Timeout set to expire in 5 min 0 sec [2021-07-26T17:32:03.046Z] [INFO] getVaultSecret: Getting secrets [2021-07-26T17:32:03.107Z] Masking supported pattern matches of $VAULT_ADDR or $VAULT_ROLE_ID or $VAULT_SECRET_ID [2021-07-26T17:32:03.551Z] [INFO] sendDataToElasticsearch: sending data... [2021-07-26T17:32:05.426Z] [INFO] getVaultSecret: Getting secrets [2021-07-26T17:32:05.460Z] Masking supported pattern matches of $VAULT_ADDR or $VAULT_ROLE_ID or $VAULT_SECRET_ID [2021-07-26T17:32:05.711Z] [INFO] sendDataToElasticsearch: sending data... [2021-07-26T17:32:06.249Z] [INFO] getVaultSecret: Getting secrets [2021-07-26T17:32:06.281Z] Masking supported pattern matches of $VAULT_ADDR or $VAULT_ROLE_ID or $VAULT_SECRET_ID [2021-07-26T17:32:06.549Z] [INFO] sendDataToElasticsearch: sending data... [2021-07-26T17:32:06.720Z] Running on Jenkins in /var/lib/jenkins/workspace/Beats_beats-tester-mbp_PR-201 [2021-07-26T17:32:06.745Z] [INFO] getVaultSecret: Getting secrets [2021-07-26T17:32:06.778Z] Masking supported pattern matches of $VAULT_ADDR or $VAULT_ROLE_ID or $VAULT_SECRET_ID [2021-07-26T17:32:07.342Z] + chmod 755 generate-build-data.sh [2021-07-26T17:32:07.342Z] + ./generate-build-data.sh https://beats-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/Beats/beats-tester-mbp/PR-201/ https://beats-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/Beats/beats-tester-mbp/PR-201/runs/13 FAILURE 1102863 [2021-07-26T17:32:07.342Z] INFO: curl https://beats-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/Beats/beats-tester-mbp/PR-201/runs/13/steps/?limit=10000 -o steps-info.json [2021-07-26T17:32:08.686Z] INFO: curl https://beats-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/Beats/beats-tester-mbp/PR-201/runs/13/tests/?status=FAILED -o tests-errors.json ```