Open adrian-herscu opened 7 years ago
Ok, I have not completely understood what you want to achieve and how should JGiven actually behave. - -- Do you want to repeat the same scenario multiple times or do you only want to repeat a single step? I ask, because your first attempt looks like that you wanted to execute a single test (=scenario) multiple times
I am looking for a way to repeat a step, or a sequence of steps, sometimes in parallel. Currently I have in my scenario:
when().repeating_$_times(2,
new FunctionWithDescription<>(
"requesting OAuth token for " + CLIEND_ID,
action -> action.requesting_OAuth_Token(
OAuthTokenRequest.builder()
.grantType(GrantType.CLIENT_CREDENTIALS)
.clientId(CLIEND_ID)
.build())));
In the report it looks like:
When repeating 2 times requesting OAuth token for zQyJJI8eRMuwSUyVYTdbKg
However, it would be nicer to have it render the repeated step itself, like this:
When repeating 2 times requesting OAuth Token
connecting to https://....
and appending path /oauth/token
and posting {client_id=[zQyJJI8eRMuwSUyVYTdbKg],grant_type...]
We have to test a log gathering system that should work in real time with huge amounts of data. Hence by design it may drop logs. This means that the tests should have some degree of tolerance. We cannot just trigger X logs and then verify that X logs where gathered. We have to trigger a relatively high number of logs and then verify that at least one was gathered.
While triggering logs, it would help running this in parallel and thus generating more logs per time unit and shortening test execution time.
This may match the design of Loop Controller from JMeter
I am sorry, but I fear I still have not completely understood what you want to achieve. Is it that you want to have a step repeated X times, but the step should only appear once in the report?
Yes. The repeated block should only appear once in the report. And the repetition might be required to occur in parallel. And the repetition might be required to occur for a sequence of steps.
Ok, thanks, I think I have finally understood the point :-)
Some questions:
Possible solution for 4: JGiven could try to detect repeating patterns/blocks in a scenario and collapse these patterns in the report. For cases with 1000 of repetitions it would actually also make sense to have a special way of representing them in the JSON model, to save space. For single steps that are immediately repeated this should be quite easy, as one could just add a counter to the step. It becomes much more difficult for blocks of steps. The only useful thing would be to apply this to blocks of nested steps.
Example (warning not tested):
when().requesting_the_OAuth_Token(100);
In the stage class:
@Nested
public SELF requesting_the_OAuth_Token(@Hidden int repetitions) {
for (int i = 0; i < repetitions; i++) {
this.connecting_to(...);
this.appending_path(...);
this.posting(...);
}
}
In the report:
when requesting the OAuth Token [x100]
connecting to ...
appending path
posting
Is the block identical for all repetitions?
Yes. Think of a block in a "for" loop or a closure in a functional language.
What should happen if there occurs an error in one of the repetitions?
The test should fail.
How should this be represented in the report?
When repeating xxx times in parallel
step 1
step 2....
How could JGiven be told that it should collapse multiple steps/blocks of steps?
This one I do not understand :)
Regarding the last point: Maybe the user does not want that steps are collapsed, so it might be that the user must tell JGIven somehow that for the following steps the repetitions should be collapsed.
I think that a good enough general solution would be that JGiven should render Java lambda expression blocks. Currently, it just calls toString() and plots that into the report.
This would open the possibility to implement generic steps such as
Now, one has to code the same loop, or try-catch and build a new step method just for looping or handling exceptions.
Ok. I think I understand what you mean. This will not be easy to implement, I think, but I could think of a new annotation to tell JGiven to render a lambda expression by rendering the steps that are called within the lambda as nested steps.
Maybe the general usecase would be reporting steps provided as parameters to other steps. This becomes common when testing asynchronous systems. Now I have a generic method like this:
public SELF retrying(final Function<SELF, SELF> step)
and almost every other step is implemented like this:
public SELF selecting_something(final String name) {
return retrying(step -> step
.clicking(By.xpath("//*[contains(text(),'" + name + "')]")));
}
In these cases the internals of select_something
are not reported :(
Sometimes a test is required to repeat a step or a sequence of steps in order to simulate some real behavior.
First I tried to use TestNG's invocationCount + threadPoolSize of @Test annotation. Then discovered that every invocation is rendered in the report. Having 100,000 invocations would make the report so big as it would be unusable.
Then I tried to write a step that accepts a TestNG class and repeatedly runs all its tests. Tried to implement it as follows:
However, when running it, something is not correctly initialized within JGiven. So fields that should be injected are not. It also requires a public class which also get executed and rendered independently.
Finally I tried to make a repeat step method that accepts another step method and just calls it repeatedly. Tried to implement is as follows:
This one works with two issues:
Is there a solution I am missing? This is a real scenario, I think it is worth framework support.