Add TestFixture

[kcooney]

Overview

• Add TestFixture, @Fixture and @ClassFixture
• Add TemporaryDirectory
• Update project to require JDK 8 (note that Java SE 6 reached EOL back in 2013)

---

I hereby agree to the terms of the Open Test Alliance Contributor License Agreement.

[kcooney]

@marcphilipp suggested adding this opentest4j (so it could be easily used by JUnit4 and JUnit5). I'd be interested to hear if other opentest4j supporters would find this useful (if not, perhaps I should instead create a new project under junit-team)

[kcooney]

The remaining Travis errors are from Spotless. Unfortunately, Spotless requires JDK 8 and I can't install JDK 8 on my Ubuntu laptop :-( If someone could add a commit fixing those issues I would appreciate it.

[sormuras]

@kcooney spotless applied against your fixtures branch.

[kcooney]

@sormuras thanks!!

[kcooney]

Thanks for the feedback, @marcphilipp !

Do you think this belongs in ota4j-team/opentest4j? I'm not sure if all users of opentest4j would support test fixtures. If it doesn't belong here, should it be a separate ota4j-team project, or a separate JUnit project? I'm leaning towards a separate JUnit project myself.

The only thing here that might be generic enough to belong in opentest4j are ClassWrapper and InstanceMethod. I've also wondered if I should introduce some kind of test/class level storage (similar to what was created for JUnit5), and that might also belong in opentest4j (so, for example, a class-level test fixture could start a server and an instance-level test fixture could connect to that server without requiring the test to wire together the two fixtures).

[leonard84]

I'm not convinced that this is a good fit for opentest4j, since it requires all frameworks to match their extension points to these constructs.

[kcooney]

@leonard84 thanks for the feedback. Marc suggested I put it here since the hope was to add support for test fixtures in JUnit 4 and JUnit5. Perhaps a separate project under junit-team would be better.

Before I put more work on this, however, I'd like to get feedback as to whether this would be useful. When I wrote it, JUnit 5 didn't support any rules; now it has some limited support for rules. Still, I'm worried that we don't have a good path forward for custom rules that can be shared between JUnit 5 and JUnit, which I think would be needed for large code bases that use rules.

[leonard84]

After reading the whole code I'm not so sure what the real added value compared to the Extension concept of JUnit Jupiter is. It is basically another level of abstraction, but the users will have to write their fixture-implementation anyway. And once https://github.com/junit-team/junit5/issues/497 is implemented, you can implement Rule-like extensions as well.

[kcooney]

@leonard84 Can JUnit4-style tests use Extension? If not, then large code bases will have problems converting from JUnit4-style to JUnit5-style if they have custom rules used by many classes.

As for junit-team/junit5#497 the last I checked, the JUnit team didn't want to support extension points that could replace the test method call, because that would allow some bad behavior (running a test multiple times, not running a test at all, etc).

[leonard84]

@kcooney No JUnit 4 does not support extension, same as JUnit 5 Jupiter does not support @Rules in general. There is support for subset see http://junit.org/junit5/docs/current/user-guide/#migrating-from-junit4-rulesupport, but as you can see in https://github.com/junit-team/junit5/tree/master/junit-jupiter-migration-support/src/main/java/org/junit/jupiter/migrationsupport/rules/adapter it requires specialized adapter code to make them work.

And there are some open issues I see:

• How do you handle inheritance?
  • Are fixtures merged
  • do they override each other
  • or are parent fixtures not applied at all?
• How do they integrate with DI-Frameworks, e.g., Spring, Guice ... ?

I don't think that @ClassFixture/@TestFixture are a good fit for a foundation library, because fixtures are highly project specific and have different requirements.

In one of my projects we have written our own fixture system, which is controlled via annotations. You can enable fixture creation with @Fixture on a class or method, you can opt-out again with @NoFixture. @Fixture(base="standard", modifications={"addUser"}) has parameters for which base data set to use and which optional modifications to apply to it. It supports fixture caching and @DirtiesFixture analogous to Springs @DirtiesContext so that read-only tests can re-use existing fixtures (this only applies to integration tests). Furthermore it supports injection of values from the fixture into the test via annotations, e.g., @TestValue("accounts[0].users[0].username") String username.

However, as a standalone library for JUnit 4 and JUnit 5 Jupiter it might be worthwhile.