refactoror / SelBlocks

SelBlocks extension for Selenium IDE
10 stars 7 forks source link

tests for "try" are failing #16

Closed matthewkastor closed 9 years ago

matthewkastor commented 9 years ago

I don't know if the test is wrong or if there is something broken in the code. The "try" test is pretty massive. I'll go through it and break it down into individual cases so it's obvious where failures are happening, if they're really failures. This will take a bit.

matthewkastor commented 9 years ago

Currently, the test "try - try without matching catch" fails and stops the test suite from continuing. The case should just fail and the suite should move on to the next test.

After breaking up the try test into individual cases, the following tests fail:

The rest of the tests appear to be passing though. I do realize that breaking up the test may be the reason that some of these cases are failing but it is easier to see where things are going wrong if the cases just test one thing at a time.

refactoror commented 9 years ago

First of all, there are no true failures in the try test. What you are seeing is that the overall test status is being set to fail by Selenium, even though catch allows the script to continue. I have fixed that for the next release. See #11.

As for "try without matching catch", that is a negative test. I.e., it should fail if everything is working properly. Therefore it is the one test that I run manually rather than as part of the full run-through. And therefore "exitTest w/finally processing" is second-to-last, which stops the test if it is working properly.

matthewkastor commented 9 years ago

I was tinkering with reBubble and there's a spot where there's a try-finally that throws the received error, just after logging that the error is not handled anywhere. I think this is blowing things up but I don't know how to fix it, other than putting a return in the finally block. It did stop the test from crashing selenium, but I was to tired to parse what was really going on. I'd have rather been able to verify that the unhandled exception bubbled up from both the try and catch (if present) and then used some selenium command to mark the case as failed and exit the test. I just couldn't figure out what those methods were, in that context.

I have some more digging to do to get selbench to work on the server. I don't know if I'll be able to though, because it manipulates the editor object. The editor is custom for selenium ide and I don't want to translate the whole ide for the server. XD I'll look hard for some ways to make it work. It sort of does, but when I get to the while test things go crazy XD.

I'm off to the cafe. Thanks for letting me know that a lot of the try tests are supposed to be marked as failed. That helps me not try figuring out why they fail on my machine XD

refactoror commented 9 years ago

Closing this as a duplicate of https://github.com/refactoror/SelBlocks/issues/6

matthewkastor commented 9 years ago

Yes, it duplcated #6 and #11 I didn't realize it at the time. Pull request #17, to break up the try test, says it has some conflicts now. I could try to fix them or just close the request. Let me know what you'd like.