Provide access to arguments of parameterized tests in lifecycle callback methods

[larryricker]

Overview

Simple parameter resolution for strings, ints, doubles, longs works in M4, but no longer works in M5 and throws the following exception.

ParameterResolutionException: No ParameterResolver registered for parameter [java.lang.String arg0] in executable

Was previously using M4, upgraded to M5 and issue began occurring.

Example:

@ParameterizedTest (name = "{index} [{arguments}] User Flags CONDITION NAME")
@ValueSource(strings = { "dev", "sit" })
@DisplayName("{index} [{arguments}] User Flags CONDITION NAME")
public void testUseTestScenario(String testEnvironment, TestInfo info, TestReporter testReporter) throws Exception {
}

Do not get a build exception, get a runtime exception with jUnit5 with gradle 3.5 running in Jenkins.

    => org.junit.jupiter.api.extension.ParameterResolutionException: No ParameterResolver registered for parameter [java.lang.String arg0] in executable [public void com.orgname.TestSituation.setUpBeforeEach(java.lang.String,org.junit.jupiter.api.TestInfo,org.junit.jupiter.api.TestReporter) throws java.lang.Exception].
  JUnit Jupiter:TestSituation:User Flags Situation A:2 [sit] User Flags Situation A
    MethodSource [className = 'com.orgname.TestSituation', methodName = 'testSituationA', methodParameterTypes = 'java.lang.String, org.junit.jupiter.api.TestInfo, org.junit.jupiter.api.TestReporter']
    => org.junit.jupiter.api.extension.ParameterResolutionException: No ParameterResolver registered for parameter [java.lang.String arg0] in executable [public void com.orgname.TestSituation.setUpBeforeEach(java.lang.String,org.junit.jupiter.api.TestInfo,org.junit.jupiter.api.TestReporter) throws java.lang.Exception].

Related Issues

• 833

• 1139

• 1668

• 1884

Deliverables

• [ ] ...

[sbrannen]

FYI: this is a direct result of the changes performed in conjunction with #833.

[sbrannen]

Assigned to 5.0 M6 in order to make a decision about how to proceed.

[marcphilipp]

The reason is that we no longer resolve parameters from @Source annotations for @BeforeEach etc. methods. Do you need the parameter there?

[larryricker]

Yes. I need the parameter on @BeforeEach and @AfterEach for several tests for Selenium and Appium testing. My Appium test examples do not demonstrate 'need', but more like 'nice to have', but are public. https://github.com/larryricker/ProjectOrganizerAppium/ My selenium tests 'need' parameters for @BeforeEach and @AfterEach to setup accounts for the correct respective environments.

[sormuras]

@larryricker Look at how Paul is solving the WebDriver injection task: https://github.com/paul-hammant/JUnit5_DependencyInjection_WebDriver

[rendaw]

If I've understood the topic correctly, I need this as well.

I have a dozen or so complex objects that all my tests use that I set up statically. I recently identified a couple default parameters for the object creation that actually need to be tested at multiple values... My only options forward seem to be explicitly forwarding parameters to a setup function in each test case or downgrading to M4.

Also, it would be nice if it somehow worked with BeforeAll so heavy immutable fixtures don't get generated and destroyed at each test method.

[opncow]

Any news on this? Just migrated to JUnit 5 for the improved parameterized tests. "Forwarding parameters to a setup function in each test case" feels more like a step back, though. :-/

[marcphilipp]

IMHO a setup function that gets parameterized test arguments only makes sense when all test methods in a class are parameterized and use at least similar parameters.

Thus, I think it would make sense to support this when the class is parameterized instead of the method (as we plan to do in #878).

[sbrannen]

I agree with @marcphilipp's analysis.

[Tradunsky]

Reproduces on 5.1.1 and on the latest 5.2.0-M1

org.junit.jupiter.api.extension.ParameterResolutionException: No ParameterResolver registered for parameter [java.lang.String arg1] in executable 

@ParameterizedTest(name = "meaningful name and argument {3} based on argument {0} and argument {1}")
@CsvSource({
            "6", "10", "40",
            "1", "10", "90"
})
public void meaningfulTestName(String meaningfulArg1, String meaningfulArg2, String meaningfulArg3) {
}

The same with long, int

[sbrannen]

@Tradunsky, your stack trace is incomplete: in executable.

Which executable?

If it's for a @BeforeEach/@AfterEach method, then that's expected since this behavior has not changed since 5.0 GA.

[geo-m]

@Tradunsky Shouldnt it be "6, 10, 40" instead of "6","10","40" ?

[Tradunsky]

@geo-m Good catch! Works like a charm. Thanks.

@sbrannen Thank you as well

[sbrannen]

Indeed, good catch, @geo-m!

[scharfstein-dev]

I have a somewhat convoluted but (I think) valid scenario for needing parameter injection outside of a test method - either in the constructor or a @BeforeEach setup method.

My context: In our application, we use scenario tests to reproduce situations that have led to errors in the past. Such a scenario is associated with certain test data, configuration parameters, and so on. All test classes wishing to use a certain scenario must extend a specific base class which arranges setup for the scenario (like booting up the application, configuring it and feeding it the test data). The base class performs this task in a @BeforeEach method.

What I am trying to do: The scenario configurations mentioned above need to fulfill certain consistency criteria. If they are entered incorrectly, this can introduce hard-to-find test errors. The consistency criteria can be checked automatically, and that's what I'm trying to do here. Basically, I am implementing a custom scenario test for the scenario I want to check, then I am manipulating the configuration during setup (also in a @BeforeEach method). Then, if the base class setup throws an error due to detecting the inconsistent data, the consistency test passes. If the manipulation is not detected however, the original configuration must already have been inconsistent, and the user is alerted to this with a failing test.

I want to do this for all existing scenarios automatically, so I've written a parameterized test for it that does exactly this. Works very well for me with JUnit 4, but if I want to migrate it I need to know the scenario configuration in the mentioned @BeforeEach method so I can manipulate it. Later on (in the @Test method) would be too late, because the base class has already completed setup by this point.

I know this must be a bit confusing, but I hope I have been clear enough in my description. If you have any clarification requests, please let me know!

[sbrannen]

Hi @scharfstein-dev,

I know this must be a bit confusing, but I hope I have been clear enough in my description. If you have any clarification requests, please let me know!

I don't think that's especially confusing; however, I do wonder if parameterized test classes wouldn't be a better solution to your problem.

Have you seen #871 and #878?

[scharfstein-dev]

@sbrannen, that sounds good - apparently I misunderstood the concept of parameterized test classes (although I did look at the linked issues), but if they will allow me to inject fields directly, I'm all set. Thanks!

[sormuras]

Have you seen #871 and #878?

Target parameterized test classes for 5.4.0-M2?

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. Given the limited bandwidth of the team, it will be automatically closed if no further activity occurs. Thank you for your contribution.

[tobiasdiez]

This is still not possible, and would be helpful indeed.

[zcmgyu]

In my case, I want to connect and create WebDriver to multi-devices at beforeEach, to run test cases in parallel.

[Jigar2018]

Hey guys! New to junit and doing migration work from junit 4 to junit 5. I asked this question but got to know it's kind of a duplicate to this open item. Anyone can help me understand how I can achieve this? (Please check sample code here). Thanks team!

[KeatsPeeks]

It's rather infortunate that many years in, Junit 5 still can't do everything that JUnit 4 could.

You can work around this by calling a setup method manually, or creating a (complex) ParameterResolver.

[wangyihong-yvonne]

Any instruction on how to create ParameterResolver?

[sbernard31]

I agree Parameterized class feature with parameter in @BeforeEach @AfterEach would be really useful. :+1:

Any instruction on how to create ParameterResolver?

You could look at : https://stackoverflow.com/a/69265907/5088764

[m1rza-s]

I came across this issue by looking for a way to make a parametrized @BeforeAll. I'd definately appreciate this.

[uchagani]

I'll add another use case for this: I need to get the value of the parameter from a @ParameterTest so that I can do some clean up using that value in my extension. The source comes from an EnumSource.

[t1]

Have you any thoughts about the possible API? I think coming up with a good one is not as easy as one would hope, because a ParameterResolver can resolve a parameter by type, index, annotations, parameter name (if the code is compiled with the -parameters option), or anything that it could derive from the test instance or ExecutionContext, e.g. tags. But the API we need here has to work the other way around; and it's not easy to reverse such a lookup.

Maybe we can add a simple method that allows only backward resolution by type: <T> T ExtensionContext#getResolvedParameterByType(Class<T>). An extension always needs the type of a parameter, so this is the most minimalistic approach I can think of. While very limited, it is probably also the most robust way for extensions to find a parameter: the resolvers can change their means of operation, but as long as there is only one resolution for a type, test extensions can still fetch them. This obviously won't work for, e.g., string parameters, but I'm among those who try to avoid "stringly typed" programming anyway. And it's always possible to wrap such data in, e.g., a single-component record.

I think this would make a fine first step that helps most extension writers. And if it's not sufficient for other use-cases, we can later think about the details of other means of backward resolution.

[marcphilipp]

Have you any thoughts about the possible API?

Wouldn't it be sufficient to get access to an array or list of objects via ExtensionContext?

[t1]

Wouldn't it be sufficient to get access to an array or list of objects via ExtensionContext?

You mean something like List<?> ExtensionContext#getResolvedParameters() then you can filter yourself? Somewhat basic, but a very good first step.

Maybe I can find the time to create a PR.

[marcphilipp]

It might be worth returning a custom interface so we can add convenience lookup methods to it later rather than adding more methods to ExtensionContext:

interface ExtensionContext {
    // ...
    Optional<ResolvedArguments> getResolvedArguments();
}
interface ResolvedArguments {
    List<?> getRawArguments();
    // we can add lookup methods here later
}