search

coverlet-coverage / coverlet

Cross platform code coverage for .NET
MIT License
2.97k stars 386 forks source link

Return two connected conditions inconsistent coverage #1334

Open manuelamateo opened 2 years ago

manuelamateo commented 2 years ago

When returning two logically connected conditions, then the branch and line coverages are not consistent. To reproduce this, certain conditions must be met:

  1. Return the result of two logically connected conditions (AND or OR)
  2. The tests do not cover the case that the first condition is true (with OR), but cover both cases that the returned result ist true and false

I tried different implementations for this simple task and found four different code coverages. I'm not sure which one is the bug. Interestingly if instead of the first condition the second condition is never true, then the coverage of the different implementations are consistent.

See reproduction https://github.com/meggima/coverlet-reproductions TwoConditionsBranchCoverageReproduction.cs & TwoConditionsBranchCoverageReproductionFixture.cs (Version coverlet.msbuild 3.1.2)

image

daveMueller commented 2 years ago

Thanks for reporting this. 👍

daveMueller commented 2 years ago

I debugged into this once for another issue https://github.com/coverlet-coverage/coverlet/issues/1279 that might be related but I couldn't find the issue back then. I debugged into this issue here and ended up at the same place. I think our algorithem that detects branches still has an issue somewhere here CecilSymbolHelper.BuildPointsForConditionalBranch(...). Unfortunately this requires a lot of knowledge about IL and thus it's hard to find the issue. I'm currently short on time but I will dig into this again. Also every contribution and additional information that helps to fix this bug is highly appreciated.

daveMueller commented 2 years ago

What I additonally found out that this issue doesn`t occur with 3 connected conditions. grafik