search

usnistgov / ACVP

Industry Working Group on Automated Cryptographic Algorithm Validation
https://csrc.nist.gov/projects/cryptographic-algorithm-validation-program
172 stars 66 forks source link

Parallel download requests #227

Closed smuellerDD closed 6 years ago

smuellerDD commented 6 years ago

When registering, the server may respond with a number of vsIDs. I would like to implement a parallel downloading of all vsIDs. I.e. I would like to spawn multiple threads one handling one vsID. Is there a limit that I should observe to be nice to the server? I.e. which netiquette is expected with such an approach?

atvassilev commented 6 years ago

@smuellerDD, as I notified you of the latest updates on the server, the demo environment is now capable of handling multiple requests in parallel. There is no specific limit, only the available resources on the platform will determine the practical bounds of how many requests will be handled in parallel. Please give it a try and let us know what you find.

smuellerDD commented 6 years ago

Am Freitag, 2. März 2018, 03:48:14 CET schrieb Apostol Vassilev:

Hi Apostol,

@smuellerDD, as I notified you of the latest updates on the server, the demo environment is now capable of handling multiple requests in parallel. There is no specific limit, only the available resources on the platform will determine the practical bounds of how many requests will be handled in parallel. Please give it a try and let us know what you find.

I set up my system to support up to 32 download threads.

Downloading all SHA and symmetric test vectors in parallel (15 vsIds and thus 15 threads) worked flawless.

Once the other definitions (HMAC, DRBG) work I will retry -- this should allow some 30 parallel downloads.

Ciao Stephan

atvassilev commented 6 years ago

Stephan, this is good news, thanks!

atvassilev commented 6 years ago

Closing this, may reopen later if issues pop up.

smuellerDD commented 6 years ago

Register operation of 20 parallel threads was successful. Test vector submission of 22 parallel threads was successful in the sense that the server accepted all data. However, after receiving 16 verdicts, the remaining verdicts could not be retrieved as the server responded with a retry:30 from that point on.

smuellerDD commented 6 years ago

@atvassilev May I bring this topic up again. I would like to suggest that NIST makes some form of statement about the maximum number of parallel requests.

As you may know, our tool will now parallelize all vsID down/uploads as well as all test session up/downloads.

Just yesterday I finished an ACVP request definition for an IUT which will for one validation round require 316(!) test sessions which each has on average 20 vsIDs(!). And these numbers are likely to grow. Granted, this is by far the largest IUT scope we have.

I also have other IUTs which, for one validation round will have 20 test sessions for one platform with about 20 vsIDs each. Now, if we want to validate on, 2 platforms, we have 40 test sessions (and so on). Again you see that the NIST servers will be confronted with a large number of requests.

As of now, I limit the number of threads for the entire tool to 32. But this is an artificial limit. I can easily go into the thousand thread range.

As I want to be nice to the NIST servers, I would seek some guidance on what I am allowed to do.