unitaryfund / mitiq

Mitiq is an open source toolkit for implementing error mitigation techniques on most current intermediate-scale quantum computers.
https://mitiq.readthedocs.io
GNU General Public License v3.0
358 stars 157 forks source link

Codecov report score fluctuates even when no changes #2318

Closed cosenal closed 4 months ago

cosenal commented 4 months ago

Hm, do you know why codecov is complaining that this line was an "indirect change?" https://app.codecov.io/gh/unitaryfund/mitiq/pull/2312/indirect-changes

It's unrelated to this PR. What is happening is that the order in which the tests get executed affects that line. Since the order test is random, that line ends up being covered or not. In fact, if you look at the list of recent coverage reports, you can see the score oscillates between 98.29% and 98.33%. I will try to find the root cause of this "non-determinism"

Originally posted by @cosenal in https://github.com/unitaryfund/mitiq/issues/2312#issuecomment-2070925879

cosenal commented 4 months ago
Screenshot 2024-04-24 at 20 08 16

^ line that gets covered or not non-deterministically according to the order in which tests are executed.