not ok 1 - Balena supervisor updater will run on a randomized timer

klutchell commented 2 years ago

These tests are failing for generic-aarch64 with QEMU. https://jenkins.product-os.io/view/tests/job/leviathan-v2-template/952/console

leviathan-v2-template-client-1  | [2022-05-20T17:26:09.725Z][worker-cloud]         # Subtest: Update supervisor randomized timer
leviathan-v2-template-client-1  | [2022-05-20T17:26:09.726Z][worker-cloud]             not ok 1 - Balena supervisor updater will run on a randomized timer
leviathan-v2-template-client-1  | [2022-05-20T17:26:09.726Z][worker-cloud]               ---
leviathan-v2-template-client-1  | [2022-05-20T17:26:09.726Z][worker-cloud]               at:
leviathan-v2-template-client-1  | [2022-05-20T17:26:09.726Z][worker-cloud]                 line: 179
leviathan-v2-template-client-1  | [2022-05-20T17:26:09.726Z][worker-cloud]                 column: 16
leviathan-v2-template-client-1  | [2022-05-20T17:26:09.726Z][worker-cloud]                 file: /data/suite/tests/supervisor/index.js
leviathan-v2-template-client-1  | [2022-05-20T17:26:09.726Z][worker-cloud]                 type: global
leviathan-v2-template-client-1  | [2022-05-20T17:26:09.726Z][worker-cloud]               stack: |
leviathan-v2-template-client-1  | [2022-05-20T17:26:09.726Z][worker-cloud]                 /data/suite/tests/supervisor/index.js:179:16
leviathan-v2-template-client-1  | [2022-05-20T17:26:09.727Z][worker-cloud]               source: |2
leviathan-v2-template-client-1  | [2022-05-20T17:26:09.727Z][worker-cloud]                           } while (samples < 3);
leviathan-v2-template-client-1  |                           test.notOk (
leviathan-v2-template-client-1  |                 ---------------^
leviathan-v2-template-client-1  |                             nextTriggers.every( (v, i, a) => v === a[0] ),
leviathan-v2-template-client-1  | [2022-05-20T17:26:09.728Z][worker-cloud]                             'Balena supervisor updater will run on a randomized timer'
leviathan-v2-template-client-1  | [2022-05-20T17:26:09.728Z][worker-cloud]               ...
leviathan-v2-template-client-1  | [2022-05-20T17:26:09.740Z][worker-cloud] 
leviathan-v2-template-client-1  | [2022-05-20T17:26:09.741Z][worker-cloud]             Bail out! Balena supervisor updater will run on a randomized timer

klutchell commented 2 years ago

FYI @alexgg I have seen this fail in the same place 3 times now. I wonder if there is an issue generating random numbers in the aarch64 image?

klutchell commented 2 years ago

alexgg commented 2 years ago

@klutchell I was testing a different approach in https://github.com/balena-os/meta-balena/pull/2655

It uses local SSH which is more reliable and does not need to increase delays

balena-os / balena-generic

not ok 1 - Balena supervisor updater will run on a randomized timer #74