search

TypedDevs / bashunit

A simple testing library for bash scripts. Test your bash scripts in the fastest and simplest way.
https://bashunit.typeddevs.com
MIT License
327 stars 28 forks source link

Bashunit Tests Fail in Alpine #329

Closed skinner-m-c closed 1 month ago

skinner-m-c commented 1 month ago
Q A
OS Linux; Alpine 3.20.x
Shell bash
bashunit version 0.15.0

Summary

Bashunit fails execute tests in Alpine.

Current behavior

When make test run all test pass in Debian based environments, but not in Busybox environments such as Alpine.

How to reproduce

$make test
./src/clock.sh: line 6: _OS: unbound variable
make: *** [Makefile:64: test] Error 1

When I fix this error I get more errors, one related to an regular expression, such as: 

Running tests/acceptance/bashunit_upgrade_test.sh
grep: bad regex '^\s*(function)?\s*test[a-zA-Z_][a-zA-Z_0-9]*\s*\(\)?\s*{': Invalid contents of {}

When I fix this error I get more errors specific to the choices made in test and the assertion functions. For example one test make use of a non-standard awk expression.

I can provide the fixes for some of these errors, but some errors seem filesystem dependent, distribution dependent or might require some alterations to the design, such as detecting what the Linux distribution is and executing different code.

Full Test Run in Alpine ```bash localhost:~/tmp2/bashunit# cat out.txt bashunit - 0.15.0 Running tests/acceptance/bashunit_direct_fn_call_test.sh ✗ Failed: Bashunit direct fn call passes Expected '1' to be exactly '0' ✗ Failed: Bashunit direct fn call passes line 6: _OS: unbound variable ✗ Failed: Bashunit direct fn call without assert prefix passes Expected '1' to be exactly '0' ✗ Failed: Bashunit direct fn call without assert prefix passes line 6: _OS: unbound variable ✗ Failed: Bashunit assert line count Expected '1' to be exactly '0' ✗ Failed: Bashunit assert line count line 6: _OS: unbound variable ✗ Failed: Bashunit assert contains Expected '1' to be exactly '0' ✗ Failed: Bashunit assert contains line 6: _OS: unbound variable ✗ Failed: Bashunit assert not contains Expected '1' to be exactly '0' ✗ Failed: Bashunit assert not contains line 6: _OS: unbound variable ✗ Failed: Bashunit assert matches Expected '1' to be exactly '0' ✗ Failed: Bashunit assert matches line 6: _OS: unbound variable ✗ Failed: Bashunit assert not matches Expected '1' to be exactly '0' ✗ Failed: Bashunit assert not matches line 6: _OS: unbound variable ✗ Failed: Bashunit assert string starts with Expected '1' to be exactly '0' ✗ Failed: Bashunit assert string starts with line 6: _OS: unbound variable ✗ Failed: Bashunit assert string not starts with Expected '1' to be exactly '0' ✗ Failed: Bashunit assert string not starts with line 6: _OS: unbound variable ✗ Failed: Bashunit assert string ends with Expected '1' to be exactly '0' ✗ Failed: Bashunit assert string ends with line 6: _OS: unbound variable ✗ Failed: Bashunit assert string not ends with Expected '1' to be exactly '0' ✗ Failed: Bashunit assert string not ends with line 6: _OS: unbound variable ✗ Failed: Bashunit direct fn call failure Expected to match the snapshot [-✗ Failed: Main::exec assert-] [- Expected 'foo'-] [- but got 'bar'-] ✗ Failed: Bashunit direct fn call failure line 6: _OS: unbound variable./src/clock.sh: line 6: _OS: unbound variable ✗ Failed: Bashunit direct fn call non existing fn Expected to match the snapshot [-Function non_existing_fn does not exist.-] ✗ Failed: Bashunit direct fn call non existing fn Expected '1' to be exactly '127' ✗ Failed: Bashunit direct fn call non existing fn line 6: _OS: unbound variable./src/clock.sh: line 6: _OS: unbound variable Running tests/acceptance/bashunit_execution_error_test.sh ✒ Incomplete: Bashunit when a execution error Add snapshots with regex to assert this test (part of the error message is localized) ✒ Incomplete: Bashunit when a execution error Add snapshots with simple/verbose modes as in bashunit_pass_test and bashunit_fail_test ✗ Failed: Bashunit when a execution error Expected '' to contain 'Running ./tests/acceptance/fixtures/test_bashunit_when_a_execution_error.sh ✗ Failed: Error Expected '127' to be exactly '1'' ✗ Failed: Bashunit when a execution error Expected '' to contain ' Tests: 1 failed, 1 total Assertions: 1 failed, 1 total' ✗ Failed: Bashunit when a execution error line 6: _OS: unbound variable./src/clock.sh: line 6: _OS: unbound variable./src/clock.sh: line 6: _OS: unbound variable Running tests/acceptance/bashunit_fail_test.sh ✗ Failed: Bashunit when a test fail verbose output env Expected to match the snapshot [-Running ./tests/acceptance/fixtures/test_bashunit_when_a_test_fail.sh-] [-✓ Passed: Assert same-] [-✓ Passed: Assert contains-] [-✗ Failed: Assert failing-] [- Expected '1'-] [- but got '0'-] [-✓ Passed: Assert greater and less than-] [-✓ Passed: Assert empty-] [-Tests: 4 passed, 1 failed, 5 total-] [-Assertions: 6 passed, 1 failed, 7 total-] [- Some tests failed -] ✗ Failed: Bashunit when a test fail verbose output env line 6: _OS: unbound variable./src/clock.sh: line 6: _OS: unbound variable ✗ Failed: Bashunit when a test fail verbose output option Expected to match the snapshot [-Running ./tests/acceptance/fixtures/test_bashunit_when_a_test_fail.sh-] [-✓ Passed: Assert same-] [-✓ Passed: Assert contains-] [-✗ Failed: Assert failing-] [- Expected '1'-] [- but got '0'-] [-✓ Passed: Assert greater and less than-] [-✓ Passed: Assert empty-] [-Tests: 4 passed, 1 failed, 5 total-] [-Assertions: 6 passed, 1 failed, 7 total-] [- Some tests failed -] ✗ Failed: Bashunit when a test fail verbose output option line 6: _OS: unbound variable./src/clock.sh: line 6: _OS: unbound variable ✒ Incomplete: Different verbose snapshots matches The different snapshots for these tests should also be identical to each other, option to choose snapshot name? ✒ Incomplete: Bashunit when a test fail simple output env Should print something like ...F. ✒ Incomplete: Bashunit when a test fail simple output option Should print something like ...F. ✒ Incomplete: Different simple snapshots matches The different snapshots for these tests should also be identical to each other, option to choose snapshot name? Running tests/acceptance/bashunit_find_tests_command_line_test.sh ✗ Failed: All tests files within a directory Expected to match the snapshot [-Running ./tests/acceptance/fixtures/tests_path/a_test.sh-] [-✓ Passed: Assert greater and less than-] [-✓ Passed: Assert empty-] [-Running ./tests/acceptance/fixtures/tests_path/other_test.sh-] [-✓ Passed: Assert same-] [-✓ Passed: Assert contains-] [-Tests: 4 passed, 4 total-] [-Assertions: 6 passed, 6 total-] [- All tests passed -] ✗ Failed: All tests files within a directory line 6: _OS: unbound variable ✗ Failed: All tests files within a file Expected to match the snapshot [-Running ./tests/acceptance/fixtures/tests_path/a_test.sh-] [-✓ Passed: Assert greater and less than-] [-✓ Passed: Assert empty-] [-Tests: 2 passed, 2 total-] [-Assertions: 3 passed, 3 total-] [- All tests passed -] ✗ Failed: All tests files within a file line 6: _OS: unbound variable ✗ Failed: All tests files with wildcard Expected to match the snapshot [-Running ./tests/acceptance/fixtures/tests_path/a_test.sh-] [-✓ Passed: Assert greater and less than-] [-✓ Passed: Assert empty-] [-Running ./tests/acceptance/fixtures/tests_path/other_test.sh-] [-✓ Passed: Assert same-] [-✓ Passed: Assert contains-] [-Tests: 4 passed, 4 total-] [-Assertions: 6 passed, 6 total-] [- All tests passed -] ✗ Failed: All tests files with wildcard line 6: _OS: unbound variable ✗ Failed: Error when no tests found line 6: _OS: unbound variable Running tests/acceptance/bashunit_log_junit_test.sh ✗ Failed: Bashunit when log junit option Expected to match the snapshot [-Running ./tests/acceptance/fixtures/test_bashunit_when_log_junit.sh-] [-✓ Passed: Success-] [-✗ Failed: Failure-] [- Expected '2'-] [- but got '3'-] [-Tests: 1 passed, 1 failed, 2 total-] [-Assertions: 1 passed, 1 failed, 2 total-] [- Some tests failed -] ✗ Failed: Bashunit when log junit option Expected 'custom.xml' to exist but 'do not exist' ✗ Failed: Bashunit when log junit option line 6: _OS: unbound variablerm: can't remove 'custom.xml': No such file or directory ✗ Failed: Bashunit when log junit env Expected to match the snapshot [-Running ./tests/acceptance/fixtures/test_bashunit_when_log_junit.sh-] [-✓ Passed: Success-] [-✗ Failed: Failure-] [- Expected '2'-] [- but got '3'-] [-Tests: 1 passed, 1 failed, 2 total-] [-Assertions: 1 passed, 1 failed, 2 total-] [- Some tests failed -] ✗ Failed: Bashunit when log junit env Expected 'log-junit.xml' to exist but 'do not exist' ✗ Failed: Bashunit when log junit env line 6: _OS: unbound variablerm: can't remove 'log-junit.xml': No such file or directory Running tests/acceptance/bashunit_pass_test.sh ✗ Failed: Bashunit when a test passes verbose output env Expected to match the snapshot [-Running ./tests/acceptance/fixtures/test_bashunit_when_a_test_passes.sh-] [-✓ Passed: Assert same-] [-✓ Passed: Assert contains-] [-✓ Passed: Assert greater and less than-] [-✓ Passed: Assert empty-] [-Tests: 4 passed, 4 total-] [-Assertions: 6 passed, 6 total-] [- All tests passed -] ✗ Failed: Bashunit when a test passes verbose output env Expected '1' to be exactly '0' ✗ Failed: Bashunit when a test passes verbose output env line 6: _OS: unbound variable./src/clock.sh: line 6: _OS: unbound variable ✗ Failed: Bashunit when a test passes verbose output option Expected to match the snapshot [-Running ./tests/acceptance/fixtures/test_bashunit_when_a_test_passes.sh-] [-✓ Passed: Assert same-] [-✓ Passed: Assert contains-] [-✓ Passed: Assert greater and less than-] [-✓ Passed: Assert empty-] [-Tests: 4 passed, 4 total-] [-Assertions: 6 passed, 6 total-] [- All tests passed -] ✗ Failed: Bashunit when a test passes verbose output option Expected '1' to be exactly '0' ✗ Failed: Bashunit when a test passes verbose output option line 6: _OS: unbound variable./src/clock.sh: line 6: _OS: unbound variable ✒ Incomplete: Different verbose snapshots matches The different snapshots for these tests should also be identical to each other, option to choose snapshot name? ✗ Failed: Bashunit when a test passes simple output env Expected to match the snapshot [-....-] [-Tests: 4 passed, 4 total-] [-Assertions: 6 passed, 6 total-] [- All tests passed -] ✗ Failed: Bashunit when a test passes simple output env Expected '1' to be exactly '0' ✗ Failed: Bashunit when a test passes simple output env line 6: _OS: unbound variable./src/clock.sh: line 6: _OS: unbound variable ✗ Failed: Bashunit when a test passes simple output option Expected to match the snapshot [-....-] [-Tests: 4 passed, 4 total-] [-Assertions: 6 passed, 6 total-] [- All tests passed -] ✗ Failed: Bashunit when a test passes simple output option Expected '1' to be exactly '0' ✗ Failed: Bashunit when a test passes simple output option line 6: _OS: unbound variable./src/clock.sh: line 6: _OS: unbound variable ✒ Incomplete: Different simple snapshots matches The different snapshots for these tests should also be identical to each other, option to choose snapshot name? Running tests/acceptance/bashunit_path_test.sh ✗ Failed: Bashunit without path env nor argument Expected to match the snapshot [-Error: At least one file path is required.-] [-bashunit [arguments] [options]-] [-Arguments:-] [- Specifies the directory or file containing the tests to run.-] [- If a directory is specified, it will execute the tests within files ending with test.sh.-] [- If you use wildcards, bashunit will run any tests it finds.-] [-Options:-] [- -a|--assert -] [- Run a core assert function standalone without a test context.-] [- --debug-] [- Print all executed shell commands to the terminal.-] [- -e|--env -] [- Load a custom env file overriding the .env environment variables.-] [- -f|--filter -] [- Filters the tests to run based on the test name.-] [- -l|--log-junit -] [- Create a report JUnit XML file that contains information about the test results.-] [- -r|--report-html -] [- Create a report HTML file that contains information about the test results.-] [- -s|simple || -v|verbose-] [- Enables simplified or verbose output to the console.-] [- -S|--stop-on-failure-] [- Force to stop the runner right after encountering one failing test.-] [- --version-] [- Displays the current version of bashunit.-] [- --upgrade-] [- Upgrade to latest version of bashunit.-] [- --help-] [- This message.-] [-See more: https://bashunit.typeddevs.com/command-line-] ✗ Failed: Bashunit without path env nor argument line 6: _OS: unbound variable./src/clock.sh: line 6: _OS: unbound variable ✗ Failed: Bashunit with argument path Expected to match the snapshot [-Running tests/acceptance/fixtures/tests_path/a_test.sh-] [-✓ Passed: Assert greater and less than-] [-✓ Passed: Assert empty-] [-Running tests/acceptance/fixtures/tests_path/other_test.sh-] [-✓ Passed: Assert same-] [-✓ Passed: Assert contains-] [-Tests: 4 passed, 4 total-] [-Assertions: 6 passed, 6 total-] [- All tests passed -] ✗ Failed: Bashunit with argument path Expected '1' to be exactly '0' ✗ Failed: Bashunit with argument path line 6: _OS: unbound variable./src/clock.sh: line 6: _OS: unbound variable ✗ Failed: Bashunit with env default path Expected to match the snapshot [-Running tests/acceptance/fixtures/tests_path/a_test.sh-] [-✓ Passed: Assert greater and less than-] [-✓ Passed: Assert empty-] [-Running tests/acceptance/fixtures/tests_path/other_test.sh-] [-✓ Passed: Assert same-] [-✓ Passed: Assert contains-] [-Tests: 4 passed, 4 total-] [-Assertions: 6 passed, 6 total-] [- All tests passed -] ✗ Failed: Bashunit with env default path Expected '1' to be exactly '0' ✗ Failed: Bashunit with env default path line 6: _OS: unbound variable./src/clock.sh: line 6: _OS: unbound variable ✗ Failed: Bashunit argument overloads default path Expected to match the snapshot [-Tests: 0 total-] [-Assertions: 0 total-] [- No tests found -] ✗ Failed: Bashunit argument overloads default path line 6: _OS: unbound variable./src/clock.sh: line 6: _OS: unbound variable Running tests/acceptance/bashunit_report_html_test.sh ✗ Failed: Bashunit when report html option Expected to match the snapshot [-Running ./tests/acceptance/fixtures/test_bashunit_when_report_html.sh-] [-✓ Passed: Success-] [-✗ Failed: Fail-] [- Expected 'to be empty'-] [- but got 'non empty'-] [-↷ Skipped: Skipped-] [-✒ Incomplete: Todo-] [-Tests: 1 passed, 1 skipped, 1 incomplete, 1 failed, 4 total-] [-Assertions: 1 passed, 1 skipped, 1 incomplete, 1 failed, 4 total-] [- Some tests failed -] ✗ Failed: Bashunit when report html option Expected 'custom.html' to exist but 'do not exist' ✗ Failed: Bashunit when report html option line 6: _OS: unbound variablerm: can't remove 'custom.html': No such file or directory ✗ Failed: Bashunit when report html env Expected to match the snapshot [-Running ./tests/acceptance/fixtures/test_bashunit_when_report_html.sh-] [-✓ Passed: Success-] [-✗ Failed: Fail-] [- Expected 'to be empty'-] [- but got 'non empty'-] [-↷ Skipped: Skipped-] [-✒ Incomplete: Todo-] [-Tests: 1 passed, 1 skipped, 1 incomplete, 1 failed, 4 total-] [-Assertions: 1 passed, 1 skipped, 1 incomplete, 1 failed, 4 total-] [- Some tests failed -] ✗ Failed: Bashunit when report html env Expected 'report.html' to exist but 'do not exist' ✗ Failed: Bashunit when report html env line 6: _OS: unbound variablerm: can't remove 'report.html': No such file or directory Running tests/acceptance/bashunit_stop_on_failure_test.sh ✗ Failed: Bashunit when stop on failure option Expected to match the snapshot [-Running ./tests/acceptance/fixtures/test_bashunit_when_stop_on_failure.sh-] [-✓ Passed: A success-] [-✗ Failed: B error-] [- Expected '1'-] [- but got '2'-] ✗ Failed: Bashunit when stop on failure option line 6: _OS: unbound variable./src/clock.sh: line 6: _OS: unbound variable ✗ Failed: Bashunit when stop on failure env Expected to match the snapshot [-Running ./tests/acceptance/fixtures/test_bashunit_when_stop_on_failure.sh-] [-✓ Passed: A success-] [-✗ Failed: B error-] [- Expected '1'-] [- but got '2'-] ✗ Failed: Bashunit when stop on failure env line 6: _OS: unbound variable./src/clock.sh: line 6: _OS: unbound variable ✒ Incomplete: Different snapshots matches The different snapshots for these tests should also be identical to each other, option to choose snapshot name? ✒ Incomplete: Bashunit when stop on failure env simple output Should print something like .F Running tests/acceptance/bashunit_test.sh ✒ Incomplete: Bashunit should display version Add snapshots with regex to assert this test (part of the output changes every version) ✗ Failed: Bashunit should display version Expected '' to contain 'bashunit - 0.15.0' ✗ Failed: Bashunit should display version Expected '1' to be exactly '0' ✗ Failed: Bashunit should display version line 6: _OS: unbound variable./src/clock.sh: line 6: _OS: unbound variable ✗ Failed: Bashunit should display help Expected to match the snapshot [-bashunit [arguments] [options]-] [-Arguments:-] [- Specifies the directory or file containing the tests to run.-] [- If a directory is specified, it will execute the tests within files ending with test.sh.-] [- If you use wildcards, bashunit will run any tests it finds.-] [-Options:-] [- -a|--assert -] [- Run a core assert function standalone without a test context.-] [- --debug-] [- Print all executed shell commands to the terminal.-] [- -e|--env -] [- Load a custom env file overriding the .env environment variables.-] [- -f|--filter -] [- Filters the tests to run based on the test name.-] [- -l|--log-junit -] [- Create a report JUnit XML file that contains information about the test results.-] [- -r|--report-html -] [- Create a report HTML file that contains information about the test results.-] [- -s|simple || -v|verbose-] [- Enables simplified or verbose output to the console.-] [- -S|--stop-on-failure-] [- Force to stop the runner right after encountering one failing test.-] [- --version-] [- Displays the current version of bashunit.-] [- --upgrade-] [- Upgrade to latest version of bashunit.-] [- --help-] [- This message.-] [-See more: https://bashunit.typeddevs.com/command-line-] ✗ Failed: Bashunit should display help Expected '1' to be exactly '0' ✗ Failed: Bashunit should display help line 6: _OS: unbound variable./src/clock.sh: line 6: _OS: unbound variable Running tests/acceptance/bashunit_upgrade_test.sh ✗ Failed: Do not upgrade when latest Expected '> You are already on latest version' but got '' ✗ Failed: Upgrade when a new version found Expected '> Upgrading bashunit to latest version' to contain '> bashunit upgraded successfully to latest version 0.15.0' ✗ Failed: Upgrade when a new version found Expected 'bashunit - 0.1.0' to end with '0.15.0' bashunit - 0.1.0 ✗ Failed: Do not update on consecutive calls Expected '> You are already on latest version' but got '> Upgrading bashunit to latest version' ✗ Failed: Do not update on consecutive calls Expected 'bashunit - 0.1.0' to end with '0.15.0' Running tests/acceptance/fixtures/tests_path/a_test.sh ✓ Passed: Assert greater and less than ✓ Passed: Assert empty Running tests/acceptance/fixtures/tests_path/other_test.sh ✓ Passed: Assert same ✓ Passed: Assert contains Running tests/acceptance/install_test.sh ✗ Failed: Install downloads the latest version Expected './lib/bashunit' to exist but 'do not exist' ✗ Failed: Install downloads the latest version Expected '' to start with 'bashunit - ' ✗ Failed: Install downloads in given folder Expected './deps/bashunit' to exist but 'do not exist' ✗ Failed: Install downloads in given folder Expected '' to start with 'bashunit - ' ✗ Failed: Install downloads the given version Expected './lib/bashunit' to exist but 'do not exist' ✗ Failed: Install downloads the given version Expected 'bashunit - 0.9.0' but got '' ✓ Passed: Install downloads the non stable beta version Running tests/acceptance/mock_test.sh ✓ Passed: Runner clear mocks first ✓ Passed: Runner clear mocks second Running tests/functional/custom_asserts_test.sh ✓ Passed: Assert foo passed ✓ Passed: Assert foo failed ✓ Passed: Assert positive number passed ✓ Passed: Assert positive number failed Running tests/functional/logic_test.sh ✓ Passed: Text should be equal ✓ Passed: Text should contain ✓ Passed: Text should not contain ✓ Passed: Text should match a regular expression ✓ Passed: Text should not match a regular expression ✓ Passed: Should validate an ok exit code ✓ Passed: Should validate a non ok exit code ✓ Passed: Other way of using the exit code Running tests/functional/provider_test.sh ✓ Passed: Multiple values from data provider (aa bb) ✓ Passed: Multiple values from data provider (aa bb) ```

Expected behavior

I expect tests to pass and bashunit to operate in environments where non-standard extensions to CLI utilities are not available. I notice that the github actions run in only Ubuntu environments. It might make sense to include other distributions of Linux to ensure bashunit works in the widest possible Linux distributions.

Chemaclass commented 1 month ago

Thanks for reporting this. Do you mind:

Chemaclass commented 1 month ago

Done https://github.com/TypedDevs/bashunit/pull/331