Add new built-in keyword "Require test case"

[GoogleCodeExporter]

I would like a new keyword, "Require test case" (or perhaps "Require" or 
"Depends on") which will fail if the given test case was not run, or was run 
and failed.

While it's never good to write test cases that depend on other test cases, the 
reality is that people will sometimes create this situation. By introducing 
this keyword, these dependencies can be exposed and enforced, and resulting 
failures will be easier to identify.

For example, imagine the following suite:

    *** Test Cases ***
    | Verify we can create a new account
    | | Create a new account | Foo

    <some other test cases>

    | Verify we can delete a new account
    | | Delete the account | Foo

As written, it's not obvious that the last test depends on the first. If the 
first test fails, the later one may fail with a message such as "Account foo 
doesn't exist" which is only half true. Arguably a more useful message would be 
"Required test case "Verify we can create a new account" failed". This keyword 
lets you know that the true root cause is the test that creates the account 
failed. As a bonus, keywords will fail quicker when a dependency isn't met. 

With this new keyword, the test case would look like:

    | Verify we can delete a new account
    | | Require test case | Verify we can create a new account
    | | Delete the account | Foo

While not ideal, it at least exposes poor practices rather than sweeping them 
under the rug.

This keyword is fairly straight forward to implement, all of the data needed to 
implement it is publicly exposed (depending on how you define "publicly").  
I've attached a .py file with my first attempt at writing it. It only supports 
dependencies within a suite, since dependencies betweens suites should always 
be strongly discouraged (though inter-suite dependencies could be supported if 
need be)

While this keyword is easy enough for me to add locally, I think including it 
to BuiltIn will be useful to others. Plus it guarantees that the keyword it 
will continue to work if/when the underlying data structures change.

As a bonus, this might help pave the way to a smarter test runner some day -- 
it could build dependency chains in order to split a test suite into parallel 
test runs. 

Original issue reported on code.google.com by bryan.oa...@gmail.com on 14 May 2014 at 9:30

Attachments:

• require_test_case.py

[GoogleCodeExporter]

I don't want this to be included in Robot standard libraries for following 
reasons:

1) I don't want to encourage users to create dependencies between tests. I know 
you cannot always avoid them, but having built-in functionality like this would 
make it look like it is a good practice.

2) If the usage is changed so that tests that are dependencies for others 
register themselves, this is easy to implement using custom keywords. This 
approach actually has two nice benefits compared to handling dependencies 
automatically:

a) It is explicit which test case is a dependency. That is important when 
editing such a test later.

b) Single test can have multiple dependency points. It could register a 
dependency when e.g. when an account is created and again when the user is 
activated.

Original comment by pekka.klarck on 15 May 2014 at 9:49

• Changed state: WontFix

[GoogleCodeExporter]

If you're concerned about encouraging bad behavior, you could have robot 
generate a warning whenever that keyword was used. You could provide a command 
line option to silence the warning, which means the user has to explicitly 
acknowledge that using that keyword is a bad practice (eg: 
--no-warn-on-bad-practices)

I don't  understand the idea of a test registering itself as a dependency for 
something else. That means if you write a new test case that depends on an old 
test case, you must also modify that old test case to register itself.  That 
seems like more work for the tester.

I understand your concerns. It's just that in my experience now in my third 
company using Robot, dependencies are a fact of life. I'd rather document them 
then have them hidden from view. Plus, the information can be invaluable when 
trying to parallelize a test run. 

Original comment by bryan.oa...@gmail.com on 15 May 2014 at 10:29