athena-oss / athena

An automation platform with a plugin architecture that allows you to easily create and share services.
https://athena-oss.github.io/athena/
Apache License 2.0
92 stars 25 forks source link

ensure function to test really exists #13

Closed ghost closed 8 years ago

ghost commented 8 years ago

in case of a typo or mistake done by the test writer, non-existing functions might generate false positives

example:

function check_input()
{
  if [ "$1" == "bad" ]; then
    return 1
  fi

  return 0
}

athena.test.assert_return.expects_fail check_ipnut bad

this was expecting the method to fail on input 'bad' but because of the typo, any change on the check_input wasn't tested and so someone might change the check_input behavior without getting a failed test

upside: this might save developers who accidentaly mistype a function or forgets to refactor a test if a function was renamed

downside: this increases the runtime for assertions a bit

ghost commented 8 years ago

with 200 rounds of athena cli tests the average increase of timing is ~1.7%

ghost commented 8 years ago

ping