Closed Zincfox closed 2 years ago
Hi @Zincfox, thanks for the comprehensive description of the problem.
I've been digging into it for a while and realized that the root cause of this strange behavior is just the way how spring tests work. If you have several test classes (including the nested ones), you should expect that each of them is being executed in a different spring context. Note that there is a caching mechanism for reusing contexts among tests, but you can't count on it. So in your case, you have three different test classes with potentially three different contexts TestContainerHost
, TestContainerImpl
and SubTest
. The first two classes share the same configuration parameters, so the context is reused in this case, the data source refers to the same database and everything works as expected. However, the nested test in the abstract class has no relation to the configuration parameters of the TestContainerHost
class. That results in the creation of a new context and this new context contains a different data source bean referring to a different database instance.
In other words, you are passing a data source bean from the TestContainerHost
context to another one with different setup and expecting the tests in SubTest
to be refreshing the passed data source, but in reality the @AutoConfigureEmbeddedDatabase
annotation on the SubTest
class is linked to and refreshes a different data source and a different database instance. You can verify it by invoking hostSource.unwrap(EmbeddedDatabase.class).getJdbcUrl()
and getting real connection details to target databases. See the attached example below.
So my recommendation is to simplify the hierarchy of test classes. Because from my point of view the @AutoConfigureEmbeddedDatabase
annotation behaves correctly, just like any other spring annotations. And for instance, the behavior of Spring's @Sql
annotation is the same in this case. The @Sql
annotation also has no idea about the passed data source and executes all statements on the undesired data source.
@Nested
@AutoConfigureEmbeddedDatabase(refresh = AutoConfigureEmbeddedDatabase.RefreshMode.BEFORE_EACH_TEST_METHOD)
@TestMethodOrder(MethodOrderer.OrderAnnotation.class)
class SubTest {
@Autowired
private DataSource dataSource;
@Test
@Order(-5)
public void selectElementsBeforeAllInsertTests() throws SQLException {
assertSelectResults(hostSource, 1, 2, 3);
}
@RepeatedTest(4)
@Order(0)
public void insertElementTest(RepetitionInfo info) throws SQLException {
// the following two lines should print different connection string to different database instances
System.out.println(hostSource.unwrap(EmbeddedDatabase.class).getJdbcUrl());
System.out.println(dataSource.unwrap(EmbeddedDatabase.class).getJdbcUrl());
int insertValue = info.getCurrentRepetition() + 3;
try (Statement insertStatement = hostSource.getConnection().createStatement()) {
Assertions.assertFalse(insertStatement.execute(
"INSERT INTO app.demo_table VALUES (" + insertValue + ");"),
"INSERT reported results instead of update-count");
Assertions.assertEquals(1, insertStatement.getUpdateCount());
}
assertSelectResults(hostSource, 1, 2, 3, insertValue);
}
@Test
@Order(5)
public void selectResetElementsAfterAllInsertTests() throws SQLException {
assertSelectResults(hostSource, 1, 2, 3);
}
}
Thank you, that makes a lot of sense to me. And I agree that this sounds way more like a 'me' problem and not something that needs fixing on your end, so I'll go ahead and close this issue for now.
At least in my concrete usecase (not the simplified one I posted) I think inheriting nested tests still allows for a more structured codebase, so I will probably tinker around with this a bit more and if all else fails stick with the workaround of nested implementations for abstract nested tests.
If I can find a better solution for my workaround I'll post it here so that others encountering this problem can hopefully save themselves some hassle :)
I believe that when nested tests are discovered through inheritance (abstract
A
has nestedN
, subclassS
inherits nestedN
) the nested tests do not respectRefreshMode.BEFORE_EACH_METHOD
.The original project where I encountered this was a kotlin-spring project with Flyway and the R2DBC adapter from #121, though I could reproduce this issue with jdbc (see below). There it could be seen that as long as the nested tests do not modify the database, all nested tests can access the migrated and filled-with-data (via
@Sql(scripts=...)
) database as expected. But once one test is added that modifies the data, all tests after it operate only the migrated version of the database - until the test execution leaves the nested class, where refreshing resumes before each test as intended. It later turned out that the database only contains data when another test of the Nest-Hosting subclass is executed before the nested ones - if not, the nested classes receive only the migrated-but-not-filled database.In our tests we passed references to the
ApplicationContext
between the nesting-levels instead of using spring-injection, which could be considered bypassing spring and as such might be an important detail in this issue.The only workaround I could discover was making the nested classes (
A.N
) in the abstract host-class abstract as well and then explicitly subclass them in the subclass again (S
extendsA
,A.N
is nested inA
,S.N
extendsA.N
,S.N
is nested inS
), which however defeats the purpose of inheriting the nested classes. Adding@Sql
annotations to the nested classes in the abstract class appear to be executed against the unmigrated database (our schema cannot be found and its tables do not exist in the default schema either), even when@AutoConfigureEmbeddedDatabase
and@FlywayTest
are added as well.Sadly I have had a lot of trouble reducing this problem out of our project-code, as such the reproducer I wound up with is still a bit long:
Which results in the following test-results:
Where repetition 2 retains the new element from repetition 1, repetition 3 those from rep. 2 and 1, rep. 4 of rep. 3, 2 and 1, and
selectResetElementsAfterAllInsertTests
operates on the data left by repetition 4.I have created a gist where I added other files that could be relevant to this problem (
build.gradle
,application.properties
etc.)The behavior with "if no outside tests are executed before the nested tests" can be seen when deleting the
NestedTestBaseExample.TestContainerHost.TestContainerImpl.OutsideTest()
test - this results in not only all tests starting with the second iteration ofinsertElementTest
failing, but those before that as well as the database contains no data.There is one more 'gotcha' I would like to point out because it caused me a lot of headache while trying to reproduce this: It is important that the
org.junit.jupiter.api.Test
-Annotation is used because theorg.junit.Test
-Annotation appears to not support (inherited?) nested testing and as such will report that no tests could be found in this case.