Restart Results Issues

[davecornwell58]

When checking the restart results I noticed that the Most Common Value entry was repeated twice. I also noticed that some of the restart results did not agree with the NIST 90B restart results that I had independently run on the same data. At the end of the restart results listing the entry for H_r H_c H_I min(H_r, H_c, H_I) and the verdict was also missing.

The following shows the repeated entry.

restart-entropy-rate.json:

{

 "id": 1114,
 "status": "RunSuccessful",
 "results": [

   {

     "dataWordSize": 8,
     "h_c": 7.8603230912910194,
     "h_r": 7.8603230912910194,
     "retMinEntropy": 7.8603230912910194,
     "testCaseDesc": "Estimate entropy with Most Common Value"

   },
   {

     "dataWordSize": 8,
     "h_c": 7.8603230912910194,
     "h_r": 7.8603230912910194,
     "retMinEntropy": 7.8603230912910194,
     "testCaseDesc": "Estimate entropy with Most Common Value"

   }, 

thanks Dave Cornwell

[davecornwell58]

further update... When comparing the returned restart results with the 90B test tool results, several of the H_c results are repeated between the different statistical test methods. Some examples are given below. The min entropy results do not seem to make sense either. { "dataWordSize": 8, "h_c": 7.3537577331212258, "h_r": 7.3337658725840837, "retMinEntropy": 7.822199513999724, "testCaseDesc": "Estimate entropy with t-Tuple Test" }, { "dataWordSize": 8, "h_c": 7.3537577331212258, "h_r": 6.4422752193931636, "retMinEntropy": 7.822199513999724, "testCaseDesc": "Estimate entropy with LRS Test" }, { "dataWordSize": 8, "h_c": 7.3537577331212258, "h_r": 6.4422752193931636, "retMinEntropy": 7.9349836056441605, "testCaseDesc": "Estimate entropy with Multi Most Common in Window Test" },

[mmccarl]

i also have found issues running ea_restart. in my original registration i specified bitspersample = 4, alphabetsize = 16 and hminestimate = 3.2. then i uploaded a 'bad' file (it only had 3 bits per sample with the 5 high order bits all zeros). surprisingly, i got results for the analysis, but when i ran it on my local copy of the 90b tool, it said that the symbols appear to be narrower than described then it aborted with a core dump because Assertion 'k_max <= k' failed.

here is the registration and the json outputs from the esv system and my local copy (note, my local copy does not produce json output, but you can see the parameters i supplied to ea_restart):

''' { "esvRegistration": [ { "esvVersion": "1.0" }, { "conditioningComponent": [ { "sequencePosition": 1, "vetted": false, "description": "parallel XOR-ed LFSRs with output buffer", "bijectiveClaim": false, "minNin": 16, "minHin": 4, "nw": 16, "nOut": 8 }, { "sequencePosition": 2, "vetted": true, "description": "AES-CBC-MAC", "validationNumber": "A0004", "minNin": 128, "minHin": 4, "nw": 128, "nOut": 128 } ], "itar": false, "primaryNoiseSource": "Ring Oscillators", "iidClaim": false, "physical": true, "additionalNoiseSources": false, "bitsPerSample": 4, "alphabetSize": 16, "hminEstimate": 3.2, "numberOfRestarts": 1000, "samplesPerRestart": 1000, "numOEs": 1 } ], "dataFileUrls": [ { "restartTestBits": "\/esvp\/v1\/entropyAssessments\/430\/dataFiles\/1164", "file-originalname": "3bittest.bin" } ], "esvoutput": { "type": "esvDataFileStatusResponse", "esvDataFileStatusResponse": [ { "esvVersion": "1.0" }, { "id": 1164, "status": "RunSuccessful", "results": [ { "dataWordSize": 4, "h_c": 2.9847392170898384, "h_r": 2.9847392170898384, "retMinEntropy": 2.9847392170898384, "testCaseDesc": "Estimate entropy with Most Common Value" }, { "dataWordSize": 4, "h_c": 2.9847392170898384, "h_r": 2.9847392170898384, "retMinEntropy": 2.9847392170898384, "testCaseDesc": "Estimate entropy with Most Common Value" }, { "dataWordSize": 4, "h_c": 2.8208833291571827, "h_r": 2.810615152863589, "retMinEntropy": 2.9847392170898384, "testCaseDesc": "Estimate entropy with t-Tuple Test" }, { "dataWordSize": 4, "h_c": 2.8208833291571827, "h_r": 2.7669759391292668, "retMinEntropy": 2.9847392170898384, "testCaseDesc": "Estimate entropy with LRS Test" }, { "dataWordSize": 4, "h_c": 2.8208833291571827, "h_r": 2.7669759391292668, "retMinEntropy": 2.9922850845194198, "testCaseDesc": "Estimate entropy with Multi Most Common in Window Test" }, { "dataWordSize": 4, "h_c": 2.8208833291571827, "h_r": 2.7669759391292668, "retMinEntropy": 2.993376584021765, "testCaseDesc": "Estimate entropy with Lag Prediction Test" }, { "dataWordSize": 4, "h_c": 2.8208833291571827, "h_r": 2.7669759391292668, "retMinEntropy": 2.987855027264817, "testCaseDesc": "Estimate entropy with Multi Markov Model with Counting Test (MultiMMC)" }, { "dataWordSize": 4, "h_c": 2.8208833291571827, "h_r": 2.7669759391292668, "retMinEntropy": 2.9866516260074225, "testCaseDesc": "Estimate entropy with LZ78Y Test" } ], "sha256": "5f02257449fd456f56e64ca97141fef3dd5bfca7234eba825c6991d407cdb4c6" } ] }, "localoutput", { "fileName": "5\/16\/EntropyData.2022-07-21_20_51_50.bin", "uploadedFileName": "3bittest.bin", "assessmentType": "ea_restart -n", "type": "entropyResponse", "entropyResponse": [ { "command": "\/usr\/src\/SP800-90B\/SP800-90B_EntropyAssessment\/cpp\/ea_restart -n -o '\/home\/entropy\/data\/EntropyData.2022-07-21_20_51_50.bin.json' -v '\/home\/entropy\/data\/EntropyData.2022-07-21_20_51_50.bin' '4' '3.2'", "Result": null } ] } }

[celic]

We use the same 90B tool as is available on GitHub, so this is a bit curious. When was the last time you pulled the 90B tool from GitHub, or what release of the tool are you using?

[mmccarl]

i don't want to get my observations confused with those of @davecornwell58, but i pulled and built the latest code on july 20. when i run 'ea_restart --version', it reports 'restart 1.1.3' (and a healthy amount of disclaimer).

in my previous post i put all of my relevant info into a single json that includes the registration, the response from ESV and the response when i run my local copy. i hope that's not too convoluted.

you might be able to re-create my issue if you create a 1000000 byte file that only includes byte values of \x00-\x07 (the high order bits zeroed out) then specify the 'number of bits' parameter to ea_restart as 4. (doing this on my system reports the samples are narrower than described and core dumps)

note that because the ESV system does not report the exact command that was run, i cannot tell if i'm running it the same way. the json i reported earlier does contain the command string my system used.

[celic]

I think what is happening here is that we are capturing the min(test_result, previous_min) rather than just the test_result in the JSON. All of the tests are being run, but the output is being captured after we compare it to the previous minimum rather than before. We should be able to fix this soon.

[davecornwell58]

Chris,

Has this been fixed yet?

thank you! Dave Cornwell

On 8/5/2022 3:58 PM, Chris Celi wrote:

I think what is happening here is that we are capturing the |min(test_result, previous_min)| rather than just the |test_result| in the JSON. All of the tests are being run, but the output is being captured after we compare it to the previous minimum rather than before. We should be able to fix this soon.

— Reply to this email directly, view it on GitHub https://github.com/usnistgov/ESV-Server/issues/21#issuecomment-1206809483, or unsubscribe https://github.com/notifications/unsubscribe-auth/A2FYMOLGASNRQVCIZXWOKMDVXVW45ANCNFSM54LRWKIQ. You are receiving this because you were mentioned.Message ID: @.***>

-- David Cornwell, PhD Principal Cryptographic and Security Consultant, @.*** atsec information security Fax: +1-512-615-7301 9130 Jollyville Road #260, Austin, TX 78759 Web: www.atsec.com

[celic]

Not yet but in progress. This issue isn't really a breaking problem in any way. The tests are still being run, it's just that the results being reported are after being compared to the running min(prevMinEntropy, testResult). If you run the tests locally without JSON parsing, you'll see the true results.

[celic]

@mmccarl @davecornwell58 We shipped a fix to Demo just now. Please run again.

[davecornwell58]

Hello Chris,

Everything looks good now !

thank you Dave Cornwell

On 8/23/2022 11:19 AM, Chris Celi wrote:

@mmccarl https://github.com/mmccarl @davecornwell58 https://github.com/davecornwell58 We shipped a fix to Demo just now. Please run again.

— Reply to this email directly, view it on GitHub https://github.com/usnistgov/ESV-Server/issues/21#issuecomment-1224223535, or unsubscribe https://github.com/notifications/unsubscribe-auth/A2FYMOLFT3YU2JPFUQT2WLTV2TTX5ANCNFSM54LRWKIQ. You are receiving this because you were mentioned.Message ID: @.***>

-- David Cornwell, PhD Principal Cryptographic and Security Consultant, @.*** atsec information security Fax: +1-512-615-7301 9130 Jollyville Road #260, Austin, TX 78759 Web: www.atsec.com

[celic]

Thank you for confirming. @mmccarl if you still see issues (I think you raised slightly tangential concerns) you may open another issue.