search

uboslinux / ubos-tools

UBOS development/deployment tools
7 stars 0 forks source link

Webapptest improvements #36

Closed jernst closed 3 years ago

jernst commented 3 years ago

From @ethn:

A couple of things would make the test development experience a lot better.

Some explicit indication that a test has passed would help. I think the tests are passing, but some confirmation would be nice.

Most test platforms give you a sum at the end of how many tests were run / passed / failed.

It seems from your structure that you're making the important distinction between a failure and an error, but I'm not sure it's working. Eg take this test:

$c->getMustContain('/', 'Your deck is ready to go', 200, 'Wrong front page');

It seems to pass (although nothing explicitly says so). If I changed the text of Your deck is ready to go to some other random text, I would expect a failure. But instead I'm getting an error.

jernst commented 3 years ago

Can you clarify @ethn: "making the important distinction between a failure and an error" -- what are you referring to here?

ethn commented 3 years ago

If I expect a given function to return 5 and

  1. it returns 5, that's a "success"
  2. it returns 6, that's a "failure".
  3. it blows up, that's an "error"

The distinction between 2 and 3 isn't as important as catching that something is wrong, but most testing platforms count these separately, and I often find that helpful.

jernst commented 3 years ago

Closing this, because it's been broken down into smaller parts.