GET /acvp/validation/acvp/results only returns "retry:30"

[smuellerDD]

I uploaded the test results to the server followed by a GET using

GET /acvp/validation/acvp/results?vsId=1141 HTTP/1.1

I get the following result from the server:

[ { "acvVersion" : "0.4" }, { "results" : { "vsId" : 1141, "disposition" : "incomplete" }, "retry" : 30 } ]

As documented, I keep the HTTPS channel open and retry the GET operation after 30 seconds. Yet, after 5 minutes of retries the server always tells me to wait for 30 seconds.

What is expected to do when I see that server response?

[atvassilev]

Depending on the algorithm you are trying to validate, it make take some time to complete. You should keep trying for a longer period of time.. In particular if another job submitted by a different client is also running. We have seen significant delays occur in such circumstances.

The current implementation of the server is rather non-optimal, hence slow performance on some validations. We are working to improve on this but the latest optimizations are still being tested internally and have not been deployed yet. Sorry for the inconvenience and thanks for your patience.

[smuellerDD]

Am Freitag, 23. Februar 2018, 23:58:53 CET schrieb Apostol Vassilev:

Hi Apostol,

Depending on the algorithm you are trying to validate, it make take some time to complete. You should keep trying for a longer period of time.. In particular if another job submitted by a different client is also running. We have seen significant delays occur in such circumstances.

The current implementation of the server is rather non-optimal, hence slow performance on some validations. We are working to improve on this but the latest optimizations are still being tested internally and have not been deployed yet. Sorry for the inconvenience and thanks for your patience.

Thanks for the hint. I restarted the submission of the test results and the retrieval of the test verdict from the server.

Again, I get the retry answer from the server for now 7 hours. I guess that this is not intended.

I am processing a response to a test vector set for AES-ECB with 128, 192 and 256 bits.

If it would be helpful, I could post the JSON file with the test responses.

Ciao Stephan

[smuellerDD]

Please note that even getting test vectors for a SHA-256 request only returns retry responses, even after waiting for several hours.

[atvassilev]

@smuellerDD, are you trying with your code or using the Cisco client that is referenced in the Readme?

[smuellerDD]

I am working with my own code. I the symmetric cipher vectors with it, but not the responses. I use the following JSON definition:

[{
    "acvVersion": "0.4"
  },
  {
    "operation": "register",
    "certificateRequest": "no",
    "debugRequest": "yes",
    "production": "no",
    "encryptAtRest": "yes",
    "oeInformation": {
      "vendor": {
        "name": "atsec corp.",
        "website": "www.atsec.com",
        "contact": [{
          "name": "Stephan Müller",
          "email": "smueller@atsec.com"
        }]
      },
      "module": {
        "name": "libgcrypt",
        "version": "1.8",
        "type": "software"
      },
      "operationalEnvironment": {
        "dependencies": [{
            "type": "software",
            "name": "Fedora 27",
            "cpe": "XXXXUNCLEARXXXX"
          },
          {
            "type": "processor",
            "manufacturer": "Intel",
            "family": "X86",
            "name": "Intel(R) Core(TM) i7-5557U",
            "series": "Broadwell",
            "features": [
              "rdrand",
              "aes-ni",
              "clmulni"
            ]
          }
        ]
      },
      "implementationDescription": "atsec library generic C implementations"
    },
    "capabilityExchange": {
      "algorithms": [{
        "algorithm": "SHA-256",
        "inBit": false,
        "inEmpty": true
      }]
    }
  }
]

[atvassilev]

To narrow the possibilities a bit, would it be possible for you to try the Cisco client and see if you get a different behavior?

[smuellerDD]

We just tested the libacvp / CISCO code. Apart from using debugRequest:no and certificateRequest:yes, it produces the very same JSON register request as my code.

The CISCO code also waits indefinitely for obtaining SHA values (note, we disabled all ciphers except SHA).

[celic]

Hey, there's a known issue on the server right now where if the registration causes the generation process to not terminate (almost always this is because of an error in the registration, or how the registration is parsed), the server will not report the error and will only send the retry message.

This has been fixed on our internal environment and will be deployed to the external environment soon. @hbooth might have more information on this.

[hbooth]

I was able to reproduce the issue for the SHA example. We have a number of items going through test right now and we can add this to that. I will close this issue when the fix has been pushed into our demo environment.

[smuellerDD]

Am Dienstag, 27. Februar 2018, 15:29:35 CET schrieb Harold Booth:

Hi Harold,

I was able to reproduce the issue for the SHA example. We have a number of items going through test right now and we can add this to that. I will close this issue when the fix has been pushed into our demo environment.

Thank you.

Though, I can also not obtain the test verdict after submitting the test results from the IUT. I always get a retry answer from the server, even after waiting for 7h. Would that be fixed as well?

Ciao Stephan

[hbooth]

I will need to know which specific example you are having trouble with and I can take a look and try to reproduce. I am assuming you are not talking about the SHA example in this case.

[smuellerDD]

Am Dienstag, 27. Februar 2018, 16:13:56 CET schrieb Harold Booth:

Hi Harold,

I will need to know which specific example you are having trouble with and I can take a look and try to reproduce. I am assuming you are not talking about the SHA example in this case.

attached is the example that I uploaded and waiting on for a long time.

Note, this is AES-ECB.

Ciao Stephan

[celic]

Unfortunately attachments don't work with replying to a GitHub issue via email. You may have to paste in the json via the website.

[smuellerDD]

Pasting a file that is 12000 lines in size will not really work.

[Kritner]

@smuellerDD could you post a sample of your response from a few test groups? specifically an "AFT" test group response, and an "MCT" test group response?

[smuellerDD]

Note, this is AES-ECB, so no IVs

[
  {
    "acvVersion":"0.4"
  },
  {
    "vsId":1141,
    "testResults":[
      {
        "tcId":1,
        "ct":"5fb6205dfae3ea33ab4553f9f1e74869"
      },
      {
        "tcId":2,
        "ct":"cbff0c9edc98c3f126a741bc5a667f6a15d04b67285d6bbab38974fc59faee0c"
      },
      {
        "tcId":3,
        "ct":"cf71988f8b05a960a3422d17e2974ad0f47d1947a33946eec3159c05d2ba8d521a176524a75819b94ef857b120c66761"
      },
      {
        "tcId":4,
        "ct":"e5143f411dc68dd1543e7f01c38a71dfb9dad68eb107499e1fb187861be73a448deb98fe4575f33dcba2fef8f40e61561f70dcf50c92b5bfa41e18be3161a229"
      },
...
      {
        "tcId":2144,
        "pt":"00000000000000000000000000000000"
      },
      {
        "tcId":61,
        "resultsArray":[
          {
            "key":"ad8a88dbedfcad43c93a3ba9215a4615",
            "pt":"337f70c013a59257a4ad25840556b3a7",
            "ct":"1bc847dec8619711d0c2a2a1a912d1f3"
          },
          {
            "key":"b642cf05259d3a5219f89908884897e6",
            "pt":"1bc847dec8619711d0c2a2a1a912d1f3",
            "ct":"1c768167d2da2f379dadda089e9c50e7"
          },
          {
            "key":"aa344e62f74715658455430016d4c701",
            "pt":"1c768167d2da2f379dadda089e9c50e7",
            "ct":"cd52f6e9c4df29768028f825089a1a56"
          },

[hbooth]

A fix for the SHA issue is now out on the demo site. I tried to reproduce the AES-ECB issue but couldn't do so. I am closing this particular issue, but if you can try out AES-ECB again and let us know if the problem still exists by opening a new ticket with the registration you are using and separately send a compressed version of the submission file to acvts-demo@nist.gov that would help us in tracking down the source of the issue. (Alternately, you can drag and drop the compressed file on to your comment/description and we should be able to access it that way as well.)

[smuellerDD]

There seems to be still an issue. However, that issue seems to be triggered as follows: when you send the test responses for the first time, the server accepts the data and returns the verdict. Though, when you re-submit the test vector for the same vsID (e.g. when the previous test result failed and you fixed something), the server accepts the response but never issues a verdict.

[celic]

Hi, resubmission to correct certain responses is not valid behavior for the protocol. An entirely new submission would need to take place. If this is not clear in the protocol definition we can make a new issue to address that specifically.

[smuellerDD]

Am Sonntag, 11. März 2018, 23:52:01 CET schrieb Chris Celi:

Hi Chris,

Hi, resubmission to correct certain responses is not valid behavior for the protocol. An entirely new submission would need to take place. If this is not clear in the protocol definition we can make a new issue to address that specifically.

At least it is not clear to me from reading the spec. But besides, returning an error indicator instead of leaving a client waiting indefinitely would be helpful.

Ciao Stephan

[hbooth]

There is in fact an error here that will need to be corrected. Stephan is correct that a more reasonable response should be provided and I also think that, in this case at least, his resubmission should have been accepted.

[hbooth]

actually, I shouldn't have reopened this. Opening a new issue that accurately describes this problem as this particular issue has gotten quite long. See #258.