usnistgov / ESV-Server

Entropy Source Validation Protocol and Server specifications
13 stars 12 forks source link

ea_restart output from the ESV system differs from the output produced locally #22

Closed mmccarl closed 1 year ago

mmccarl commented 2 years ago

after updating the SP800-90B tool to version 1.1.4, I ran ea_restart on my local machine using a data file that had a sample size less than the sample size specified on the command line. This causes ea_restart to fail with reason k_max <= k', no json output is produced, and the program core dumps.

next i submitted the same file to the ESV system with the same erroneous sample size and received the following result (note that dataWordSize is 4, but my input file has only 3):

[
        {
            "esvVersion": "1.0"
        },
        {
            "id": 1262,
            "status": "RunSuccessful",
            "results": [
                {
                    "dataWordSize": 4,
                    "h_c": 2.9847392170898384,
                    "h_r": 2.9847392170898384,
                    "testCaseDesc": "Most Common Value"
                },
                {
                    "dataWordSize": 4,
                    "h_c": 2.8208833291571827,
                    "h_r": 2.810615152863589,
                    "testCaseDesc": "T-Tuple Test"
                },
                {
                    "dataWordSize": 4,
                    "h_c": 2.8582530039436405,
                    "h_r": 2.7669759391292668,
                    "testCaseDesc": "LRS Test"
                },
                {
                    "dataWordSize": 4,
                    "h_c": 2.9922850845194198,
                    "h_r": 2.991042077044165,
                    "testCaseDesc": "Multi Most Common in Window Test"
                },
                {
                    "dataWordSize": 4,
                    "h_c": 2.993376584021765,
                    "h_r": 2.995140579020789,
                    "testCaseDesc": "Lag Prediction Test"
                },
                {
                    "dataWordSize": 4,
                    "h_c": 2.987855027264817,
                    "h_r": 2.9917283928740184,
                    "testCaseDesc": "Multi Markov Model with Counting Test (MultiMMC)"
                },
                {
                    "dataWordSize": 4,
                    "h_c": 2.9866516260074225,
                    "h_r": 2.992086614560301,
                    "testCaseDesc": "LZ78Y Test"
                },
                {
                    "h_c": 2.8208833291571827,
                    "h_i": 0,
                    "h_r": 2.7669759391292668,
                    "testCaseDesc": "Overall"
                }
            ],
            "sha256": "5f02257449fd456f56e64ca97141fef3dd5bfca7234eba825c6991d407cdb4c6"
        }
    ]

The sha256 in this response matches that of the file I uploaded, but i have uploaded this file before and the response may be old data. I suspect this because the 'toolVersion' element is not present in this response which matches responses I have received in the past. I have no explanation how/why I get any json output at all.

It would be nice if the json response also provided the command line that was used to run ea_restart.

I posted a related issue on the SP800-90B_EntropyAssessment repository (issue 195).

celic commented 2 years ago

This would explain some errors we had trouble tracking down on our end... Segmentation faults causing no output to be generated. It is interesting that this provides output when a similar run on just the tool doesn't. We're working on adding the tool version to this, and also it may be useful to provide the exact run command. Traceability is helpful.

celic commented 2 years ago

The next release (v1.2) will include the command being run by the entropy assessment tool.

celic commented 1 year ago

Check the command run via the server, this went to Demo last Friday. Please re-open the issue if this persists.