containers / toolbox

Tool for interactive command line environments on Linux
https://containertoolbx.org/
Apache License 2.0
2.39k stars 208 forks source link

test/system: Avoid future problems caused by Bat's 'run' overwriting 'i' #1373

Closed debarshiray closed 9 months ago

debarshiray commented 9 months ago

Until Bats 1.10.0, run with options had a bug where it would overwrite the value of the i variable even outside run [1].

In these particular instances, no options are being passed to 'run', and, hence, currently there's no problem. However, in case a future commit adds an option, then it could lead to hard-to-debug problems. eg., --separate-stderr sets i to 1, --show-output-of-passing-tests sets it to 2, etc.. Therefore, depending on the flag and the loop, the loop might get terminated prematurely or run infinitely or something else.

Moreover, Bats 1.10.0 is only available in Fedora >= 39 and is absent from Fedoras 37 and 38. Therefore, it's not possible to consider this bug fixed.

Hence, it's better to preemptively work around it to avoid any future issues.

[1] Bats commit 502dc47dd063c187 https://github.com/bats-core/bats-core/commit/502dc47dd063c187 https://github.com/bats-core/bats-core/issues/726

softwarefactory-project-zuul[bot] commented 9 months ago

Build failed. https://softwarefactory-project.io/zuul/t/local/buildset/d6399c8088fe447cb7382f03be1aad64

:heavy_check_mark: unit-test SUCCESS in 8m 32s :heavy_check_mark: unit-test-migration-path-for-coreos-toolbox SUCCESS in 3m 19s :heavy_check_mark: unit-test-restricted SUCCESS in 7m 42s :x: system-test-fedora-rawhide FAILURE in 33m 04s :heavy_check_mark: system-test-fedora-38 SUCCESS in 25m 27s :heavy_check_mark: system-test-fedora-37 SUCCESS in 24m 08s

softwarefactory-project-zuul[bot] commented 9 months ago

Build failed. https://softwarefactory-project.io/zuul/t/local/buildset/f1a0ad76af964b22b5ae5bd10e543393

:heavy_check_mark: unit-test SUCCESS in 8m 36s :heavy_check_mark: unit-test-migration-path-for-coreos-toolbox SUCCESS in 3m 31s :heavy_check_mark: unit-test-restricted SUCCESS in 7m 32s :x: system-test-fedora-rawhide FAILURE in 51m 13s :x: system-test-fedora-38 TIMED_OUT in 40m 30s :x: system-test-fedora-37 TIMED_OUT in 40m 31s

debarshiray commented 9 months ago

recheck

softwarefactory-project-zuul[bot] commented 9 months ago

Build failed. https://softwarefactory-project.io/zuul/t/local/buildset/612b5c68644c4cadb4780ce05cc934b7

:heavy_check_mark: unit-test SUCCESS in 8m 23s :heavy_check_mark: unit-test-migration-path-for-coreos-toolbox SUCCESS in 4m 14s :heavy_check_mark: unit-test-restricted SUCCESS in 7m 40s :x: system-test-fedora-rawhide FAILURE in 34m 57s :heavy_check_mark: system-test-fedora-38 SUCCESS in 27m 34s :heavy_check_mark: system-test-fedora-37 SUCCESS in 27m 36s

softwarefactory-project-zuul[bot] commented 9 months ago

Build failed. https://softwarefactory-project.io/zuul/t/local/buildset/06e61d9881f042dd9bef4e7d7d5302dd

:heavy_check_mark: unit-test SUCCESS in 8m 24s :heavy_check_mark: unit-test-migration-path-for-coreos-toolbox SUCCESS in 3m 33s :heavy_check_mark: unit-test-restricted SUCCESS in 7m 34s :x: system-test-fedora-rawhide FAILURE in 32m 14s :heavy_check_mark: system-test-fedora-38 SUCCESS in 28m 28s :heavy_check_mark: system-test-fedora-37 SUCCESS in 24m 48s

debarshiray commented 9 months ago

There are still some test failures on Fedora Rawhide. For example:

fedora-rawhide | not ok 3 help: Run command 'help' in 145ms
fedora-rawhide | # (from function `assert_line' in file test/system/libs/bats-assert/src/assert.bash, line 479,
fedora-rawhide | #  in test file test/system/002-help.bats, line 45)
fedora-rawhide | #   `assert_line --index 0 --partial "toolbox(1)"' failed
fedora-rawhide | # /usr/bin/man
fedora-rawhide | #
fedora-rawhide | # -- line does not contain substring --
fedora-rawhide | # index     : 0
fedora-rawhide | # substring : toolbox(1)
fedora-rawhide | # line      : troff:<standard input>:33: warning: cannot select font 'C'
fedora-rawhide | # --

I believe these are because of changes in various other components in Fedora 39, which we need to track down one by one and work out a fix.

In the mean time, I am going to temporarily override these failures.