jaswdr / faker

:rocket: Ultimate fake data generator for Go with zero dependencies
https://pkg.go.dev/github.com/jaswdr/faker
MIT License
549 stars 59 forks source link

[BUG] Different coverage percents every time the test run #103

Closed 4k1k0 closed 1 year ago

4k1k0 commented 2 years ago

Describe the bug If you run the test with the coverprofile flag it return different results every time.

To Reproduce

$ for i in {1..10}; do go test -coverprofile=coverage.out .; done

E.g.

$ for i in {1..10}; do go test -coverprofile=coverage.out .; done
ok      faker   6.565s  coverage: 95.6% of statements
ok      faker   5.586s  coverage: 94.9% of statements
ok      faker   6.202s  coverage: 95.6% of statements
ok      faker   5.363s  coverage: 96.2% of statements
ok      faker   5.253s  coverage: 96.4% of statements
ok      faker   5.439s  coverage: 95.9% of statements
ok      faker   5.360s  coverage: 96.3% of statements
ok      faker   5.089s  coverage: 96.2% of statements
ok      faker   6.661s  coverage: 95.1% of statements
ok      faker   9.051s  coverage: 96.2% of statements

Expected behavior It should return the same value every time.

Desktop (please complete the following information):

xlanor commented 2 years ago

This is mainly because of the fact that a ton of logic here actually relies on randoming numbers which means that there are different paths that are tested in the coverage each time.

jaswdr commented 1 year ago

I set the minimum coverage to 95%. Although all methods are exercised in tests, reaching 100% of coverage will not be possible until we have a major rewrite allowing mocking of many methods using interfaces. For now, I believe this is sufficient, but I'm more than happy to review this in the future if we receive more concerns.