search

colcon / colcon-lcov-result

A colcon extension for collecting lcov output
http://colcon.readthedocs.io
Apache License 2.0
7 stars 5 forks source link

Support for alternate backends #27

Open ruffsl opened 3 years ago

ruffsl commented 3 years ago

Over on ros-planning/navigation2 we've been encountering some recent heavy performance regressions in processing our code coverage reports. From what previously took about 3m, to now what recently takes up to +30m. We tried swapping to colcon lcov-result, as noted by a commit in the PR below, but the time performance improvements was only marginal. For further details:

https://github.com/ros-planning/navigation2/pull/2391

We are not yet sure what the original root cause for the performance regression was, but we found a nice alternative to mediate the issue by swapping out our original calls to lcov and making use of fastcov instead to speed up nav2 CI:

https://github.com/RPGillespie6/fastcov

What took about 30 minutes now only takes about 30 seconds! I figured this kind of improvement would be advantageous for other colcon build tool users. Also, by opening this ticket with folks more familiar with lcov, perhaps we could get to the bottom of the current performance regression.

CC: @SteveMacenski @nuclearsandwich @RPGillespie6 @christophebedard

christophebedard commented 3 years ago

Weird! I'll look at my CI logs/try manually and see if I see the same performance issue.

Support for alternate backends

Is this something you would be interested in contributing? E.g. a --fastcov option to use that as the backend.

christophebedard commented 3 years ago

Also, there is an open issue on https://github.com/colcon/colcon-core about writing a "colcon universal coverage result tool" which is kind of relevant: https://github.com/colcon/colcon-core/issues/359

SteveMacenski commented 3 years ago

As far as I can tell from a quick look at docs, fastcov is just lcov but parallelized for the files to speed things up. I don't actually see much, if any, reason to not simply replace it to use fastcov in general

christophebedard commented 3 years ago

As far as I can tell from a quick look at docs, fastcov is just lcov but parallelized for the files to speed things up. I don't actually see much, if any, reason to not simply replace it to use fastcov in general

Good point.

I was going to say that lcov is available through apt while fastcov is only available through PyPI, but colcon-lcov-result doesn't even declare a dependency on lcov.

However, I guess we can assume it's probably installed whereas fastcov would need to be installed manually. I think it's not a huge blocker though.

ruffsl commented 3 years ago

Is this something you would be interested in contributing? E.g. a --fastcov option to use that as the backend.

If I manage to get a seperate coverage issue resolved, then I could spare some time to help.

but colcon-lcov-result doesn't even declare a dependency on lcov.

I think it's using gcov:

https://github.com/RPGillespie6/fastcov/blob/86b2cee04cc09f02ba4be17cff78ca1e41c61867/fastcov.py#L264

which must come with build-essential or something by default in apt, as we don't explicitly install it in our Dockerfile.