failSame, failNotsame and failNotEqual functions not working properly

[GoogleCodeExporter]

I observed that the following functions are not working properly in the
2.1.5 version of shUnit2:

- failNotEquals
- failSame
- failNotSame

AFAIK, failNotEquals should fail when given parameters are not equal so
passing '1' and '0' should fail, while '1' and '1' should not. However I
always get failure, regardless of the parameters:

ASSERT:fail: not equals expected:<1> but was:<1>
ASSERT:fail: not equals expected:<1> but was:<0>

Same happens with testFailSame. This is what I tried...

failSame "failSame FAILED" "text" "text"
failSame "failSame FAILED" "text" "not same text"

...and this is what I got:

ASSERT:failSame FAILED expected not same
ASSERT:failSame FAILED expected not same

And, at last, same happens with testFailNotSame. I tried this...

failNotSame "failNotSame FAILED" "text" "text"
failNotSame "failNotSame FAILED" "text" "not same text"

...and I got this:
ASSERT:failNotSame FAILED expected:<text> but was:<text>
ASSERT:failNotSame FAILED expected:<text> but was:<not same text>

Perhaps I'm misunderstanding something, but I imagine this could be a bug,
so here you are the report. If not, I'd be very happy to know why, so I
could understand the desired behaviour better, so thanks in advance ;-)

Otherwise, please take a look into the patch I'm attaching (done against
2.1.5 version of shunit2, from googlecode SVN), I'm using it right now and
it seems to work fine
:-)

At last, just to say I've submitted this same bug in the old bugzilla at SF
(http://sourceforge.net/tracker/index.php?func=detail&aid=2062264&group_id=18800
1&atid=923569),
before realizing of googlecode being the current right place to do it, so
excuse the "spam"... 

However, as the bug seems to be there yet, I updated my patch (formerly
made against 2.0.4 vversion) according to the last code from 2.1.5 revision
of shUnit2, and that's what I'm uploading right now. Hope it helps.

Br,

Mario

Original issue reported on code.google.com by mariospr...@gmail.com on 20 Aug 2008 at 4:34

Attachments:

• fail-functions.patch

[GoogleCodeExporter]

The patch does not include fixes to the unit tests. As such, this patch fails
testing. Could you please resubmit the patch, but one that fixes the unit tests 
as well?

Original comment by kate.a.w...@gmail.com on 30 Aug 2008 at 4:29

[GoogleCodeExporter]

(In reply to comment #1)

> The patch does not include fixes to the unit tests. As such, this patch fails
> testing. Could you please resubmit the patch, but one that fixes the unit 
tests as 
> well?

Sure, but first let me ask you a doubt about the documentation of such those 
failXXX
functions, since it's a bit confusing to me and I'd prefer to be sure before 
sending
another incorrect or incomplete patch to you :-).

From
http://shunit2.googlecode.com/svn/trunk/source/2.1/doc/shunit2.html#shelldoc-sec
tion-failures:

-failSame([message]):Indicate test failure because arguments were not the same. 
The
message is optional.

Is this ok? I'd understand that this function should fail when arguments were 
the
same, and not the other way around

-failNotsame([message],expected,actual): Fails the test if expected and actual 
are
equal to one another. The message is optional.

Is this ok? Because of the same reason explained for failSame, I'd understand 
this
function should fail when arguments were NOT the same, and not the other way 
around.

In other words, I would say the description for failSame and failNotSame is 
swapped,
isn't it?

Thanks in advance.

Mario

Original comment by mariospr...@gmail.com on 1 Sep 2008 at 11:40

[GoogleCodeExporter]

As I myself am not really sure either, I will look into PyUnit, and perhaps 
some xUnit documentation on what the 
actual functionality should be. You are correct that the documentation does not 
match the functionality, so one 
or the other needs to be fixed.

If you happen across some good documentation for another unit testing framework 
that is clear on this, I'd be 
glad to look at it.

Original comment by kate.war...@gtempaccount.com on 1 Sep 2008 at 10:49

• Changed state: Accepted

[GoogleCodeExporter]

Looking at some xUnit documentation on Single Outcome Assertions 
(http://xunitpatterns.com/Assertion%20Method.html#Single%20Outcome%20Assertion) 
I found a section that 
talks about failNotEquals. "The method failNotEquals is a Test Utility Method 
that fails the test and provides a 
diagnostic assertion message."

Basically, shUnit2 is doing the correct thing in that the failNotEquals is not 
an assertion, meaning it is not 
testing that the values are equal or not. It is supposed to provide a nice 
message explaining that the values 
are not equal, but it never actually tests them. They have a code sample in 
Java that clearly demonstrates its 
usage, so I will look at porting that sample to shell and including it in the 
documentation.

Original comment by kate.war...@gtempaccount.com on 1 Sep 2008 at 11:02

[GoogleCodeExporter]

Hmmm... So after all it seems the current shunit2 is working properly, that is, 
just
always failing regardless of the passed values being equal or not... therefore 
the
only bug in shunit2 would be in the documentation :-)

IMO, this is pretty clear as far as I understand from the following excerpt 
(from
http://xunitpatterns.com/Assertion%20Method.html#Single%20Outcome%20Assertion)

   /**
    * Asserts that two objects are equal. If they are not
    * an AssertionFailedError is thrown with the given message.*/
   static public void assertEquals(String message, Object expected, Object actual) {
      if (expected == null && actual == null)
         return;
      if (expected != null && expected.equals(actual))
         return;
      failNotEquals(message, expected, actual);
   }

It seems the failX functions are no more than private helper functions, but the 
real
test of the values must be performed previously. In fact, in JUnit the 
failNotEquals
function is a static private function indeed: 

http://www.javaresearch.org/source/junit3.8.1/junit/framework/Assert.java.html

So, it seems there's no need of any patch after all, and that I just 
misunderstood
the meaning of the functions, so excuse me for the "noise" :-)... and thanks a 
lot
for your work in shUnit2: I'm using it from some weeks in my day-a-day work and 
it
works like a charm!

Original comment by mariospr...@gmail.com on 2 Sep 2008 at 7:44

[GoogleCodeExporter]

No worries! And I don't mind the noise as it required me to actually do the 
research to verify why JUnit did what 
it did. I hadn't understood that before either believe it or not.

I'm leaving this bug open until I have fixed the documentation.

Original comment by kate.a.w...@gmail.com on 7 Oct 2008 at 11:19

[GoogleCodeExporter]

Fixed in revision# 246.

Original comment by kate.war...@gtempaccount.com on 24 Oct 2008 at 12:59

• Changed state: Done
• Added labels: Type-Documentation
• Removed labels: Type-Defect