Open ViseLuca opened 4 months ago
Are you running elixir 17 on erlang 27 by any chance? I've started seeing some unexpected changes to excoveralls output after upgrading erlang and elixir.
So... I'm wrong about the elixir/erlang versions being our culprit. It turns out that we have some hidden dependencies within our test suite such that running mix test --cover
twice without changing anything, the code does take different paths and correctly reports different coverage. For example, a test that hits this function will have "flaky" coverage reporting. (ours aren't this, but for the sake of example)
def is_this_covered do
if :rand.uniform(2) == 1 do
covered_half_the_time()
else
covered_the_other_half_of_the_time()
end
end
Hi, We have a project that since yesterday in the CI/CD pipeline, and in local env, changed the coverage percentage.
mix coveralls
from 89,2% is 88.9%mix coveralls.html
is remaining 89,2% with all the testsIn my local env is changing if I run it 10 times 7 are correct, 3 are not
There is a way to discover why is changing sometimes?