openflagr / flagr-archived

[Archived] - Merged and move to https://github.com/openflagr/flagr

https://openflagr.github.io/flagr

Apache License 2.0

33 stars 7 forks source link

Add distribution threshold test #35

Closed zhouzhuojie closed 3 years ago

zhouzhuojie commented 3 years ago

Description

To make sure that the distribution result from eval flags are as close as expected with the hashing function that flagr is using (i.e. CRC32). In these tests, we are targeting 1e6 samples and a 0.5% error bar (usually it's less than 0.1%, e.g. 0.0436% from the secure random key generator, but for reducing the flakiness of unit tests, we are raising the threshold to 0.5%).

Motivation and Context

Address some of the concerns raised from https://github.com/checkr/flagr/issues/464

How Has This Been Tested?

unit tests

Types of changes

[ ] Bug fix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to change)

Checklist:

[x] My code follows the code style of this project.
[ ] My change requires a change to the documentation.
[ ] I have updated the documentation accordingly.
[x] I have added tests to cover my changes.
[x] All new and existing tests passed.

codecov-commenter commented 3 years ago

Codecov Report

Merging #35 (0f17204) into main (f161c56) will not change coverage. The diff coverage is n/a.

@@           Coverage Diff           @@
##             main      #35   +/-   ##
=======================================
  Coverage   81.55%   81.55%           
=======================================
  Files          27       27           
  Lines        2174     2174           
=======================================
  Hits         1773     1773           
  Misses        314      314           
  Partials       87       87

Continue to review full report at Codecov.

Legend - Click here to learn more Δ = absolute <relative> (impact), ø = not affected, ? = missing data Powered by Codecov. Last update f161c56...0f17204. Read the comment docs.