Improve Smart Testing Performance

arquillian / smart-testing

We know which tests you want to run.

Apache License 2.0

25 stars 12 forks source link

Improve Smart Testing Performance #236

Open dipak-pawar opened 7 years ago

dipak-pawar commented 7 years ago

Run Smart-Testing with any real project.
Find all areas which impacts execution time/performance using profiling.
After finding all areas, find optimized solutions for it.

MatousJobanek commented 7 years ago

I have one idea how to improve the performance: The calculations (scm) would be executed in a separated thread, so the main thread wouldn't be blocked and could continue with the Maven build. I guess that calculation should be probably finished before the surefire execution starts (in most of the cases), but this has to be managed anyway (in cases when it takes more time than the compilation etc...). To avoid the state when the surefire starts before the calculation is finished and tries to read non-existing or uncomplete scm-changes file, the logic should use some marker. This marker can be a temporary empty file that would represent that the execution is still being processed. When the calculation execution is finished, the file is removed. On the surefire side, this file would be checked if whether still exists or not. If not, then it can start looking/reading the scm-changes file and continue with normal test execution.

lordofthejars commented 7 years ago

Yes it is a good idea, even we could create some kind of chain on this. For example when scm changes has been calculated, if affected strategy is enabled, start calculating graph too. But maybe this is something to take into consideration as next step.

MatousJobanek commented 7 years ago

@lordofthejars IIRC you have some big project with a lot of test classes. Is it AssertJ or something else?

MatousJobanek commented 7 years ago

if affected strategy is enabled

If we don't want to detect which strategies are activated in the maven extension phase, then we cannot create this kind of chain. Instead of chaining the strategy specific tasks, we could introduce another level of optimization - running the activated strategy implementations in parallel.

lordofthejars commented 7 years ago

Yes it was just assertj

bartoszmajsak commented 7 years ago

@lordofthejars can you share modified project so we can have a simple reference to reproducer?

lordofthejars commented 7 years ago

There were no modification, I just cloned the assertj project and modified org.assertj.core.util.Strings_join_Test with one redundant test

bartoszmajsak commented 7 years ago

And how did you execute the build? Which version of ST was defined in extensions.xml?

lordofthejars commented 7 years ago

In the git history of workshop you can see next things if you want to reproduce exactly the same thing:

Affected strategy: https://github.com/lordofthejars/smart-testing-workshop/commit/f36cf842747b50f9f5cf7155ebbe774bac7d370b

You need to run this using affected strategy and it is where the 2 million nodes are created.

Changed: https://github.com/lordofthejars/smart-testing-workshop/commit/951b7c576ce149c93d203d689dd77c0edf82d5c9

You need to run this using changed strategy, but no performance issues observed at that time.

bartoszmajsak commented 7 years ago

Thanks, that's exactly the context we will need when we dive into this analysis!