patch: Add vcdbg binary to tests

[vipulgupta2048]

From the discussion: https://balena.zulipchat.com/#narrow/stream/345889-balena-io.2Fos/topic/Debugging.20device-tree.20tests.3A.20vcdbg.20alternatives

The Google Drive link for downloading vcdbg have been hitting download limit and hence failing the tests. Hence, the binary has been moved to the tests to be used locally rather than fetched every test run. Long Term: We want to migrate off this deprecated tool in favor of vclog

Signed-off-by: Vipul Gupta (@vipulgupta2048) vipulgupta2048@gmail.com

[acostach]

I don't think this is going to work because the vcdbg binary is in the worker, but is then executed in the hostOS of the DUT, where isn't present.

Also, https://github.com/balena-os/meta-balena/pull/3364/commits/7b2d71f1a79fbda5def8aca6c8a68605bf933194#diff-2ac1b9d57eb61a563121558fd17d3cc77037115448c75119696bbcf8becfacd9R239 executes vcdbg from the DUT's /tmp/

[rcooke-warwick]

@acostach you are correct. The test as it is assumes the binary is on the DUT. Whereas with this solution, its with the test suites in the core container.

You must send it to the DUT like this: https://github.com/balena-os/meta-balena/blob/master/tests/suites/hup/suite.js#L232C1-L232C80

Also @vipulgupta2048 I'd recommend trying the solution on a pi3, marking this as a draft until we confirm it works. This test won't be run on the virtualised tests on this PR

[acostach]

Thanks @rcooke-warwick , I was looking for an example like that. @vipulgupta2048 if you don't have a board to test or don't have the bandwidth, just let me know

[acostach]

@rcooke-warwick @vipulgupta2048 the above patch made the test pass on my side using the semi-automated branch and a pi3-64.

leviathan-client-1  | [2024-02-14T12:22:11.623Z][worker-os]     # Subtest: Device Tree tests
leviathan-client-1  | [2024-02-14T12:22:11.623Z][worker-os]         # Subtest: DToverlay & DTparam tests
leviathan-client-1  |             # Sending vcdbg to DUT
leviathan-client-1  | [2024-02-14T12:22:11.624Z][worker-os] GET http://worker/dut/ip
leviathan-client-1  | [2024-02-14T12:22:12.231Z][worker-os] vcdbg was successfully to the target
leviathan-client-1  | [2024-02-14T12:22:12.232Z][worker-os] using local ip: 192.168.0.129
leviathan-client-1  | [2024-02-14T12:22:14.702Z][worker-os]             ok 1 - vcdbg static binary should be downloaded and run successfuly
leviathan-client-1  | [2024-02-14T12:22:14.703Z][worker-os] using local ip: 192.168.0.129
leviathan-client-1  | [2024-02-14T12:22:14.708Z][worker-os] (node:127) DeprecationWarning: is() is deprecated, use equal() instead
leviathan-client-1  | (Use `node --trace-deprecation ...` to show where the warning was created)
leviathan-client-1  | [2024-02-14T12:22:16.477Z][worker-os]             ok 2 - vcdbg logs should be clean of device-tree or overlay load failures
leviathan-client-1  | [2024-02-14T12:22:16.482Z][worker-os]             1..2
leviathan-client-1  | [2024-02-14T12:22:16.491Z][worker-os]         ok 1 - DToverlay & DTparam tests # time=4894.033ms
leviathan-client-1  | [2024-02-14T12:22:16.492Z][worker-os] 
leviathan-client-1  | [2024-02-14T12:22:16.495Z][worker-os]         1..1
leviathan-client-1  | [2024-02-14T12:22:16.496Z][worker-os]     ok 3 - Device Tree tests # time=4918.95ms
leviathan-client-1  | [2024-02-14T12:22:16.496Z][worker-os] 

[acostach]

@resin-jenkins retest this please

[vipulgupta2048]

Thanks for your reviews folks @rcooke-warwick @acostach It has been a minute since I have written tests and missed the worker/core execution scenario. I was making the changes but running into the "configured authentication method failed" error when testing the changes locally that you requested. Should have pushed sooner and accepted the help instead, thanks for making the patch @acostach

You did it exactly how I tried, the only suggestion I would recommend doing is... actually I just made another commit to your patch with changes I would hope to make this even better. Let me know if these work on your testbot? And, it should be good to go.

[rcooke-warwick]

passed on pi4 DUT

client_1  | [2024-02-15T09:02:27.451Z][59415ad-os]     # Subtest: Device Tree tests
client_1  | [2024-02-15T09:02:27.452Z][59415ad-os]         # Subtest: DToverlay & DTparam tests
client_1  |             ok 1 - Pin 4 is High as expected
client_1  | [2024-02-15T09:02:27.464Z][59415ad-os] (node:135) DeprecationWarning: true() is deprecated, use ok() instead
client_1  | [2024-02-15T09:02:31.277Z][59415ad-os]             ok 2 - DToverlay & DTparam configured successfully
client_1  | [2024-02-15T09:02:33.661Z][59415ad-os] No response from server
client_1  | [2024-02-15T09:02:34.159Z][59415ad-os] Connection lost before handshake
client_1  | [2024-02-15T09:02:35.012Z][59415ad-os] Connection lost before handshake
client_1  | [2024-02-15T09:02:55.687Z][59415ad-os] Timed out while waiting for handshake
client_1  | [2024-02-15T09:03:48.132Z][59415ad-os]             # Waiting for DUT to come back online after reboot...
client_1  | [2024-02-15T09:03:48.133Z][59415ad-os]             # Waiting for supervisor to be ready after reboot...
client_1  |             # Waiting for supervisor to be ready after reboot...
client_1  |             ok 3 - Pin 4 set to Low after applying dtoverlay
client_1  | [2024-02-15T09:03:48.606Z][59415ad-os]             ok 4 - DToverlay successfully set in target state
client_1  | [2024-02-15T09:03:48.607Z][59415ad-os]             ok 5 - DTparam successfully set in target state
client_1  | [2024-02-15T09:03:51.343Z][59415ad-os]             ok 6 - DToverlays successfully configured in config.txt
client_1  | [2024-02-15T09:03:54.295Z][59415ad-os]             ok 7 - DTparams successfully configured in config.txt
client_1  | [2024-02-15T09:04:02.867Z][59415ad-os] vcdbg was successfully to the target
client_1  | [2024-02-15T09:04:05.826Z][59415ad-os]             # Sending vcdbg to DUT 1
client_1  | [2024-02-15T09:04:05.827Z][59415ad-os]             ok 8 - vcdbg static binary runs successfuly
client_1  | [2024-02-15T09:04:09.345Z][59415ad-os]             ok 9 - vcdbg logs should be clean of device-tree or overlay load failures
client_1  | [2024-02-15T09:04:09.346Z][59415ad-os]             1..9
client_1  | [2024-02-15T09:04:09.347Z][59415ad-os]         ok 1 - DToverlay & DTparam tests # time=110566.132ms
client_1  | [2024-02-15T09:04:09.347Z][59415ad-os] 
client_1  | [2024-02-15T09:04:09.347Z][59415ad-os]         1..1
client_1  | [2024-02-15T09:04:09.348Z][59415ad-os]     ok 6 - Device Tree tests # time=110569.584ms

[rcooke-warwick]

so LGTM

[acostach]

@rcooke-warwick is the Pi4 test run passed with Vipul's change above or without?

[vipulgupta2048]

@acostach I think it is with my changes because I added the attempts variable in the console message which lead to these logs with a 1 in them and I also changed the assertion number 8's message to not include the word download.

Although now I see the spelling for successfully is wrong 🙈 (it was wrong before too)

# Sending vcdbg to DUT 1
client_1  | [2024-02-15T09:04:05.827Z][59415ad-os]             ok 8 - vcdbg static binary runs successfuly

[rcooke-warwick]

@acostach @vipulgupta2048 I ran with pi4 with vipuls additional commit

[acostach]

@resin-jenkins retest this please

[rcooke-warwick]

@resin-jenkins retest this please

[acostach]

@resin-jenkins retest this please

[acostach]

@resin-jenkins retest this please

[acostach]

Hey @rcooke-warwick, this PR will need an approval from you because Vipul and I have commits in the branch

[vipulgupta2048]

@balena-ci I self-certify!

[vipulgupta2048]

Self-certification is still working on balena-os org it seems @acostach