Can't exclude packages using coverage.list

intersystems / TestCoverage

Test Coverage Tool

MIT License

9 stars 8 forks source link

Can't exclude packages using coverage.list #16

Closed mdavidovichHst closed 2 years ago

mdavidovichHst commented 2 years ago

Hi there, I have tried doing this multiple ways on coverage.list but I am still getting coverage report data for the HS package and sub-packages and also the TestCoverage package. Here's a snap of my coverage.list:

And here is the output on Jenkins:

I've tried: -HS -HS.PKG -HS.FHIR -HS.FHIR.PKG . . . and so on.

I know I can't use the wildcard for packages but I've tried excluding both the top level package and the exact sub-package with and without the .PKG marker.

Have I missed a key step or is this a error in the tool?

Thanks!

mdavidovichHst commented 2 years ago

Just to mention, I believe this is making the test coverage run very slow. Jenkins reports over two minutes for the report to produce. Trying to get this time down by excluding unnecessary classes.

mdavidovichHst commented 2 years ago

Issue resolved by ONLY listing includes.

mdavidovichHst commented 2 years ago

Sorry have to reopen: I'm almost there but now I notice that I have a package with about 12 sub-packages. The high-level package Interface is tracked but only three of the subpackages are tracked. All subpackages have class files in them.

Any way to fix this?

isc-tleavitt commented 2 years ago

@mdavidovichHst can you paste in the coverage.list?

isc-tleavitt commented 2 years ago

Another thought - might you have multiple files named coverage.list in your unit test directory structure that might be conflicting?

isc-tleavitt commented 2 years ago

re: subpackages not being tracked, if they were just (e.g.) Ensemble message classes with properties and no methods, there'd be nothing to track code coverage on so they wouldn't show up.

mdavidovichHst commented 2 years ago

Another thought - might you have multiple files named coverage.list in your unit test directory structure that might be conflicting?

No just the one file in the root!

@mdavidovichHst can you paste in the coverage.list?

// Include all application code
AD.PKG
Core.PKG
DataShare.PKG
DefaultAcceptMessageService.PKG
EN.PKG
HX.PKG
Interface.PKG
LB.PKG
PR.PKG
RP.PKG
Studio.PKG
SY.PKG
User.PKG

// Exclude
-PR.UnitTest.PKG
-RP.UnitTest.PKG
-LB.UnitTest.PKG

re: subpackages not being tracked, if they were just (e.g.) Ensemble message classes with properties and no methods, there'd be nothing to track code coverage on so they wouldn't show up.

Ok this makes sense and is confirmed. However there's a class with no properties and one instance method. Would that be tracked?

isc-tleavitt commented 2 years ago

@mdavidovichHst it depends on the method. A method that returns a constant ends up getting handled entirely in the class descriptor - there's no .INT code, so we can't track coverage of such methods using the line-by-line monitor. (Theoretically we could with some more invasive instrumentation, but I'd like to avoid that.)

So the short answer is, we can only track methods for which we have .INT code.