search

STAMP-project / test-runner

test driver to run JUnit tests isolated in a new JVM
GNU General Public License v3.0
16 stars 15 forks source link

Optimize coverage analysis #111

Open andre15silva opened 3 years ago

andre15silva commented 3 years ago

Hot spots of the test-runner process started by flacoco on math_70:

Self times pic-selected-210729-1739-13

Expanded self times for the top result. This highlights that the most time consuming operation is analyzing the instrumented classes after execution. pic-selected-210729-1739-56

Total times pic-selected-210729-1740-26

Where to optimize?

  1. Coverage analysis for sure. This is the most time consuming operation with ~77% of cpu time being used there. (169s/218s)

(This is copied from https://github.com/SpoonLabs/flacoco/issues/74)

andre15silva commented 3 years ago

A possible path to optimize is to use online instrumentation (See https://www.eclemma.org/jacoco/trunk/doc/agent.html)

andre15silva commented 3 years ago

FYI, I'm experimenting with online instrumentation with JMX.

See https://github.com/andre15silva/test-runner/tree/online-prototype, working prototype but it is slower from my tests.

andre15silva commented 3 years ago

Btw @danglotb, I got online instrumentation working but it was even slower, so I gave up on that.

You might want to read the last two comments of https://github.com/SpoonLabs/flacoco/issues/74, if you want to keep up to date with the optimization effort.