The meaning of ea_iid output results

mcugood commented 3 years ago

Background: I currently have 1 million RNG random numbers, use this tool to calculate whether the random numbers are enough

I want to know what these results represent, how much or how big the minimum entropy needs to be to meet the specifications. Please let me know, thank you

cmd

./cpp/ea_iid -i -a -v ./filename

result Opening file: './RNG_data_125_2.txt' Loaded 1632400 samples of 16 distinct 7-bit-wide symbols Number of Binary samples: 11426800

Samples have been translated Calculating baseline statistics... Raw Mean: 70.563127 Median: 8.000000 Binary: false

Literal MCV Estimate: mode = 107671, p-hat = 0.06595871110022053, p_u = 0.066459117709374321 Bitstring MCV Estimate: mode = 5733203, p-hat = 0.50173303111982359, p_u = 0.50211402866821697 H_original: 3.911389 H_bitstring: 0.993913 min(H_original, 7 X H_bitstring): 3.911389

Chi square independence score = 250.511662 degrees of freedom = 240 p-value = 0.307382

Chi square goodness of fit score = 129.442114 degrees of freedom = 135 p-value = 0.618777

** Passed chi square tests

LiteralLongest Repeated Substring results P_col: 0.0625481 Length of LRS: 10 Pr(X >= 1): 0.7051 ** Passed length of longest repeated substring test

Beginning initial tests...

Initial test results excursion: 11948.5 numDirectionalRuns: 1.08416e+06 lenDirectionalRuns: 9 numIncreasesDecreases: 867186 numRunsMedian: 816406 lenRunsMedian: 21 avgCollision: 4.70577 maxCollision: 15 periodicity(1): 101823 periodicity(2): 102505 periodicity(8): 102454 periodicity(16): 102170 periodicity(32): 102255 covariance(1): 8.12875e+09 covariance(2): 8.12852e+09 covariance(8): 8.12779e+09 covariance(16): 8.12794e+09 covariance(32): 8.12767e+09 compression: 888454

Beginning permutation tests... these may take some time 15.34% of Permutuation test rounds, 100.00% of Permutuation tests

            statistic  C[i][0]  C[i][1]  C[i][2]

            excursion    1528       0       6
   numDirectionalRuns       6       0       6
   lenDirectionalRuns       3       6       0
numIncreasesDecreases      11       0       6
        numRunsMedian       6       0      19
        lenRunsMedian       4       2      10
         avgCollision       6       0      11
         maxCollision       2       6       0
       periodicity(1)      15       0       6
       periodicity(2)       6       0      23
       periodicity(8)       5       1      72
      periodicity(16)       9       0       6
      periodicity(32)       6       0      31
        covariance(1)       6       0      56
        covariance(2)       6       0      16
        covariance(8)      14       0       6
       covariance(16)       6       0       9
       covariance(32)       6       0       6
          compression       6       0      10

(* denotes failed test)

** Passed IID permutation tests

celic commented 2 years ago

The IID assessment here can only confirm that a known independent and identically distributed entropy source is producing IID data. If that is the case, then the entropy estimate is roughly -log(p_max). See Section 6.3.1 of SP 800-90B for more information.

If your source is not known to be IID, you should run the ./ea_non_iid program which will produce clearer estimates of entropy.

celic commented 2 years ago

SP 800-90B does not set a minimum amount of entropy needed from an entropy source. The value determines how many bits you will need to provide to a SP 800-90A DRBG to achieve the desired security strength. The draft of SP 800-90C has more information on this topic. This draft is a bit out of date, there isn't an official SP 800-90C yet. https://csrc.nist.gov/CSRC/media/Publications/sp/800-90c/draft/documents/sp800_90c_second_draft.pdf

usnistgov / SP800-90B_EntropyAssessment

The meaning of ea_iid output results #167