Need an all-inclusive test suite and need it to pass cleanly

[GoogleCodeExporter]

Why are there no test cases to verify expected behavior of the various
collections classes or is that part of the codebase that doesnt get synced
during the integration with google's internal repo?

Original issue reported on code.google.com by TrashH...@gmail.com on 29 Mar 2008 at 6:36

[GoogleCodeExporter]

When Kevin and Jared open sourced the collections, it required much effort to 
clean up the code and to 
decouple it from from Google's other libraries. The effort to do likewise for 
the tests is underway. 

When it does eventually come out, you'll find the test coverage to be very 
cool. There's testers that can test an 
arbitrary implementation of say, Set, plus customizations of those testers for 
special cases (ie. ImmutableSet). 
It makes the test infrastructure very reusable - if we write a new 
implementation of Set, we get a lot of test 
coverage for free. 

Plus, we can also check subtle consistency issues with Sun's Java APIs. One 
example - what happens if you call 
remove(null) on Collections.emptyList()? Does it fail with an 
UnsupportedOperationException? Or does it 
return 'false' to indicate that no change was made. Either answer could be 
correct, depending on perspective. 
The collections tests cover several subtle details like this so we can keep 
track of the subtlest of behaviour 
changes.

Original comment by limpbizkit on 29 Mar 2008 at 8:25

[GoogleCodeExporter]

Original comment by kevin...@gmail.com on 27 May 2008 at 6:57

• Added labels: 1.0

[GoogleCodeExporter]

Tests are there now.  They are not passing cleanly yet, so I'll leave this open.

Original comment by kevin...@gmail.com on 17 Mar 2009 at 5:22

• Changed title: Need an all-inclusive test suite and need it to pass cleanly

[GoogleCodeExporter]

I did a fresh checkout, and then "ant test" which has a few problems.  First, 
test
needs to depend on testfw.  After testfw has run successfully, test still fails 
with
this message:

$ ant test
Buildfile: build.xml

compile:

test.compile:

test:
    [junit] Usage: TestRunner [-wait] testCaseName, where name is the name of the
TestCase class
    [junit] Picked up JAVA_TOOL_OPTIONS:  
    [junit] Picked up _JAVA_OPTIONS:  

BUILD FAILED
c:\users\projects\google-collections\build.xml:65: Java returned: 2

Original comment by motlin on 8 Apr 2009 at 2:59

[GoogleCodeExporter]

Original comment by cpov...@google.com on 18 Apr 2009 at 12:53

• Changed state: Accepted

[GoogleCodeExporter]

I've added the missing Ant dependency pointed out in comment 4 (thanks), 
included the
required cglib dependency for EasyMock classextension, and introduced a 
test-suite
class.  The tests now run, but some do not pass:

1) NullPointerTester failures.  I'm not sure what the problem here is.
2) Ordering.reverse().reverse().  The code and tests are not in sync.  This 
should be
fixed in the next snapshot.

I should also verify that all tests are being run.

Original comment by cpov...@google.com on 20 Apr 2009 at 6:59

[GoogleCodeExporter]

The problem with the NullPointerTester tests was that jsr305.jar was not in the
runtime dependencies of the tests.  I should have realized this earlier, but:

1) "foo instanceof Nullable" silently returns false if the class Nullable is 
unknown.
 This makes a little sense:  If Nullable hasn't been loaded, it's impossible for an
existing object to be an instance.  But it's not the fail-fast behavior I'd
subconsciously assumed.

2) Not all NullPointerTester tests are failing.  JoinerTest, for example, 
passes. 
The tester examines Joiner.join(@Nullable Object, @Nullable Object, Object...) 
and
doesn't see the @Nullable annotations.  But in fact Joiner.on(","),join(null, 
null,
new Object[0]) *does* throw NPE, so the test passes.  This is because the 
@Nullable
annotations indicate that, while the main Joiner implementation does not accept 
null,
subclasses may do so.  I think there may be more advanced annotations to 
indicate
this, but annotations or not, it's admittedly a bit confusing.

In any case, this is now fixed.

The only remaining issue is OrderingTest.  The public version of Ordering no 
longer
ensures that comparator.reverse().reverse().equals(comparator), but the 
corresponding
test change didn't come over along with it.  (Luckily, problems like this will 
be
easier to avoid in the future when we have a runnable test suite :))

Assigning to Kevin for him to close when he merges in the next round of 
OrderingTest
and other changes.

Original comment by cpov...@google.com on 20 Apr 2009 at 10:14

[GoogleCodeExporter]

Original comment by kevin...@gmail.com on 17 Sep 2009 at 5:45

• Added labels: Milestone-1.0

[GoogleCodeExporter]

Original comment by kevin...@gmail.com on 17 Sep 2009 at 5:46

• Removed labels: 1.0

[GoogleCodeExporter]

Original comment by kevin...@gmail.com on 17 Sep 2009 at 6:00

• Added labels: Type-Task

[GoogleCodeExporter]

Test suite has been passing cleanly for a while.

Original comment by kevin...@gmail.com on 16 Oct 2009 at 8:59

• Changed state: Fixed