usnistgov / ESV-Server

Entropy Source Validation Protocol and Server specifications
13 stars 12 forks source link

"RunSuccessful" outputs missing data #20

Closed mmccarl closed 1 year ago

mmccarl commented 2 years ago

None of the entopy analysis ouputs resulting from dataFile requests include the version of the tool (it should be 1.1.3). see ids 1038, 1039 and 1040.

also, the sha256 field for analysis of conditioned data includes an empty string for the sha256 field (see id 1040 on esv demo). this appears to be from the 90B tool.

analysis of restart data showed only the sha256 -- none of the other information was given (see id 1039 on esv demo).

following are the responses for each of the above ids;

     [
        {
            "esvVersion": "1.0"
        },
        {
            "id": 1038,
            "status": "RunSuccessful",
            "results": [
                {
                    "hBitstring": 0.9979219753078928,
                    "hOriginal": 3.9770025768882586,
                    "mcvEstimateMode": 62879,
                    "mcvEstimatePHat": 0.062879,
                    "mcvEstimatePU": 0.06350427050803746,
                    "testCaseDesc": "Estimate entropy with Most Common Value"
                },
                {
                    "hBitstring": 0.9412885063133121,
                    "testCaseDesc": "Estimate entropy with Collision Test (for bit strings only)"
                },
                {
                    "hBitstring": 0.9997312558121291,
                    "testCaseDesc": "Estimate entropy with Markov Test (for bit strings only)"
                },
                {
                    "hBitstring": 0.8876121119407032,
                    "testCaseDesc": "Estimate entropy with Compression Test (for bit strings only)"
                },
                {
                    "binTTupleRes": 0.9294337363877716,
                    "tTupleRes": 3.6489097327341495,
                    "testCaseDesc": "Estimate entropy with t-Tuple Test"
                },
                {
                    "binLrsRes": 0.998030533079676,
                    "lrsRes": 3.872577932559578,
                    "testCaseDesc": "Estimate entropy with LRS Test"
                },
                {
                    "hBitstring": 0.9975326766025671,
                    "hOriginal": 3.986552620861951,
                    "testCaseDesc": "Estimate entropy with Multi Most Common in Window Test"
                },
                {
                    "hBitstring": 0.9998086192511981,
                    "hOriginal": 3.7798743187813284,
                    "testCaseDesc": "Estimate entropy with Lag Prediction Test"
                },
                {
                    "hBitstring": 0.9984205151285463,
                    "hOriginal": 3.9751521794400375,
                    "testCaseDesc": "Estimate entropy with Multi Markov Model with Counting Test (MultiMMC)"
                },
                {
                    "hBitstring": 0.998622644353119,
                    "hOriginal": 3.9759057563290647,
                    "testCaseDesc": "Estimate entropy with LZ78Y Test"
                },
                {
                    "dataWordSize": 4,
                    "hAssessed": 3.550448447762813,
                    "hBitstring": 0.8876121119407032,
                    "hOriginal": 3.6489097327341495,
                    "testCaseDesc": "Overall"
                }
            ],
            "sha256": "71658f28dcbb3e1a26cf8ad3d28e029b414639c1afe6860d6079216a1c894acd"
        }
    ]
    [
        {
            "esvVersion": "1.0"
        },
        {
            "id": 1039,
            "status": "RunSuccessful",
            "sha256": "31120ea9bb6c4aebeca07466501545fd498893ae64af285e5ba89be4f8bc11d1"
        }
    ]
    [
        {
            "esvVersion": "1.0"
        },
        {
            "id": 1040,
            "status": "RunSuccessful",
            "results": [
                {
                    "h_in": 4,
                    "h_out": 3.9178497401531804,
                    "h_p": 0.8955054031204854,
                    "n_in": 16,
                    "n_out": 8,
                    "nw": 16,
                    "testCaseDesc": "Overall"
                }
            ],
            "sha256": ""
        }
    ]
celic commented 2 years ago

The empty content from the restart tests is odd. I don't see that behavior at the moment.

We are also prepping a deployment at the moment, so I'm testing against that, not Demo. If it continues to happen, please follow up.

For the version of the entropy assessment tool, we always use the latest from that repository, but we can include the version. I think our intention was to include the version in the preamble version information at all times. At a minimum it can certainly appear in the JSON response.

celic commented 2 years ago

We just did a new release on Demo, please check this again if you have a chance.

mmccarl commented 2 years ago

did a second try using the same input files, but still no "toolVersion" shows up. everything looks the same. i'll take a closer look next week

mmccarl commented 2 years ago

there is still no 'toolVersion' in the json response for any of the tests. i also found additional issues with restart tests that i included in issue #21

celic commented 1 year ago

This should be fixed with the release we just put on Demo.