bats loops - only last iteration is displayed in console output

brennx0r commented 8 years ago

I am writing tests under test-kitchen that will verify that installations of packages have taken place. I was hoping to do this via providing the package names as an array and providing tests using a for loop:

#!/usr/bin/env bats

slave_pkg=( openssl openssl-devel readline-devel libxslt-devel zlib-devel glib2-devel libxml2-devel patch libffi libtool git rpm-build )

for package in "${slave_pkg[@]}"
do
  @test "Jenkins Slave :: Dependencies :: Verify RHEL $package is installed." {
    run bash -c "yum info $package | grep installed"
    [[ $output = "Repo        : installed" ]]
  }
done

Expected Result: This bats file run will result in the running of 12 tests. Observed Result: Only the last iteration through the array is captured out to console output (Look for all instance of the prefix "Jenkins Slave :: Dependencies ::"):

-----> Running bats test suite
 ✓ Jenkins Master :: Setup Prep :: Jenkins user is present on the master instance
 ✓ Jenkins Master :: JDK Installation :: Verify RHEL JDK 1.8.0 is installed.
 ✓ Jenkins Master :: JDK Installation :: Verify RHEL JDK 1.7.0 is installed.
 ✓ Jenkins Master :: JDK Installation :: Verify RHEL JDK 1.6.0 is installed.
 ✓ Jenkins Master :: Installation :: Jenkins war is present in /usr/lib/jenkins
 ✓ Jenkins Master :: Installation :: Validate login page and search for "Jenkins"
 ✓ Jenkins Master :: Installation :: Assert that "Jenkins" setup page does not appear
 - Jenkins Master :: Installation :: Validate status code for login page (skipped)
 ✓ Jenkins Master :: Installation :: Jenkins service is running
 ✓ Jenkins Master :: Configuration :: Maven Pathing :: Verify that Maven pathing config file is present on the Jenkins master.
 ✓ Jenkins Master :: Configuration :: Maven Pathing :: Verify RHEL Maven 3.0.5 is present in the file.
 ✓ Jenkins Master :: Configuration :: Maven Pathing :: Verify RHEL Maven 3.1.1 is present in the file.
 ✓ Jenkins Master :: Configuration :: Maven Pathing :: Verify RHEL Maven 3.2.5 is present in the file.
 ✓ Jenkins Master :: Configuration :: Maven Pathing :: Verify RHEL Maven 3.3.3 is present in the file.
 - Jenkins Master :: Installation :: Plugins :: analysis-core - Assert value is present on the Installed Plugins page (skipped)
 ✓ Jenkins Master :: Maven Installation :: Verify RHEL Maven 3.0.5 has been installed.
 ✓ Jenkins Master :: Maven Installation :: Verify RHEL Maven 3.0.5 install pkg has been cleaned up.
 ✓ Jenkins Master :: Maven Installation :: Verify RHEL Maven 3.1.1 has been installed.
 ✓ Jenkins Master :: Maven Installation :: Verify RHEL Maven 3.1.1 install pkg has been cleaned up.
 ✓ Jenkins Master :: Maven Installation :: Verify RHEL Maven 3.2.5 has been installed.
 ✓ Jenkins Master :: Maven Installation :: Verify RHEL Maven 3.2.5 install pkg has been cleaned up.
 ✓ Jenkins Master :: Maven Installation :: Verify RHEL Maven 3.3.3 has been installed.
 ✓ Jenkins Master :: Maven Installation :: Verify RHEL Maven 3.3.3 install pkg has been cleaned up.
 ✓ Jenkins Slave :: Dependencies :: Verify RHEL rpm-build is installed.
 ✓ Jenkins Master :: Dependencies :: git binary is found in PATH
 ✓ Jenkins Master :: Dependencies :: wget binary is found in PATH

       26 tests, 0 failures, 2 skipped
       Finished verifying <default-centos-68> (0m55.79s).

Any help that folks can provide on why this is happening would be appreciated, as I'm new to bats and may have missed a doc item that discusses nuances of loop use with tests. (I didn't see anything in the docs about this.)

ztombol commented 8 years ago

@brennx0r This is a duplicate of #136. See there why this doesn't work and what are the possible solutions.

You didn't miss anything. This has to do with how bats processes test files and it's only briefly mentioned on the wiki. Which is unfortunate. This is a top item on my long list of proposed improvements that I've been planning to post once I find the time.

Just for reference, #157 is another duplicate that tried to do the same "is the package installed" use case.

brennx0r commented 8 years ago

Thanks for the info! Closing this issue out.

sstephenson / bats

bats loops - only last iteration is displayed in console output #179