Testsuite seems to be passing, even though a specific tests fails.

[zmiklank]

          Strange behaviour.

There is an error in test/run script:

-----------------------------------------------
Running test run_pgaudit_test (starting at 2023-10-03 10:26:59+00:00) ... 
-----------------------------------------------
c644c8223c59eb64a932a2d1cd2e13e9f3bee4a1c7fd79ec69e1df6e37a788c0
Created container c644c8223c59eb64a932a2d1cd2e13e9f3bee4a1c7fd79ec69e1df6e37a788c0
test/run: line 340: $1: unbound variable
Tests finished with EXIT=1

Tests are passing:

==============================================
Tests were run for image quay.io/sclorg/postgresql-15-c9s:1
==============================================
Test cases results:

 [PASSED] for 'postgresql-container_tests' run_container_creation_tests (00:00:19)
 [PASSED] for 'postgresql-container_tests' run_general_tests (00:00:39)
 [PASSED] for 'postgresql-container_tests' run_change_password_test (00:00:07)
 [PASSED] for 'postgresql-container_tests' run_replication_test (00:00:06)
 [PASSED] for 'postgresql-container_tests' run_doc_test (00:00:00)
 [PASSED] for 'postgresql-container_tests' run_s2i_test (00:00:14)
 [PASSED] for 'postgresql-container_tests' run_test_cfg_hook (00:00:08)
 [PASSED] for 'postgresql-container_tests' run_s2i_bake_data_test (00:00:12)
 [PASSED] for 'postgresql-container_tests' run_s2i_enable_ssl_test (00:00:08)
 [PASSED] for 'postgresql-container_tests' run_upgrade_test (00:00:00)
 [PASSED] for 'postgresql-container_tests' run_migration_test (00:00:23)

Originally posted by @phracek in https://github.com/sclorg/postgresql-container/issues/399#issuecomment-1744700208

[zmiklank]

It is little bit confusing what the log line Tests finished with EXIT=1 is saying to us. In reality it means, that the exit code of the test script was not zero, however, this does not mean, that the TESTSUITE_RESULT was success or that the tests finished successfully. I do not see the results of run_pgaudit_test in the logs that Peto has found. That could mean, that unbound variable led to exiting the whole testsuite even before the test result itself was recorded. This is not really bad behaviour, unfortunately from the log I can not see, if the overal test results were green or red. Everything leads to conclusion that they were red - so I consider this behaviour to be at least partly correct.

What can we do in order to increase readibility of the logs:

• change sentence Tests finished with EXIT=1 to something like Tests finished unexpectedly with EXIT=1
• do not exit test script on unbound variable, just fail the specific testcase if this happens.

From my POV this problem does not have high priority. @phracek Do you want to add something?

[phracek]

@SlouchyButton It looks like you are the expert for this issue. As you did it in s2i-ruby-container and s2i-nodejs-container. If you do not have time, I will fix it.