LMS keyGen testvector generation failure when all LMS and LMOTS modes specified

[jbteron]

environment Demo

testSessionId 369783

vsId 1491314

Algorithm registration

{
    "algorithm": "LMS",
    "capabilities": {
        "lmOtsModes": [
            "LMOTS_SHA256_N24_W1",
            "LMOTS_SHA256_N24_W2",
            "LMOTS_SHA256_N24_W4",
            "LMOTS_SHA256_N24_W8",
            "LMOTS_SHA256_N32_W1",
            "LMOTS_SHA256_N32_W2",
            "LMOTS_SHA256_N32_W4",
            "LMOTS_SHA256_N32_W8",
            "LMOTS_SHAKE_N24_W1",
            "LMOTS_SHAKE_N24_W2",
            "LMOTS_SHAKE_N24_W4",
            "LMOTS_SHAKE_N24_W8",
            "LMOTS_SHAKE_N32_W1",
            "LMOTS_SHAKE_N32_W2",
            "LMOTS_SHAKE_N32_W4",
            "LMOTS_SHAKE_N32_W8"

        ],
        "lmsModes": [
            "LMS_SHA256_M24_H5",
            "LMS_SHA256_M24_H10",
            "LMS_SHA256_M24_H15",
            "LMS_SHA256_M24_H20",
            "LMS_SHA256_M24_H25",
            "LMS_SHA256_M32_H5",
            "LMS_SHA256_M32_H10",
            "LMS_SHA256_M32_H15",
            "LMS_SHA256_M32_H20",
            "LMS_SHA256_M32_H25",
            "LMS_SHAKE_M24_H5",
            "LMS_SHAKE_M24_H10",
            "LMS_SHAKE_M24_H15",
            "LMS_SHAKE_M24_H20",
            "LMS_SHAKE_M24_H25",
            "LMS_SHAKE_M32_H5",
            "LMS_SHAKE_M32_H10",
            "LMS_SHAKE_M32_H15",
            "LMS_SHAKE_M32_H20",
            "LMS_SHAKE_M32_H25"
        ]
    },
    "mode": "keyGen",
    "revision": "1.0"
}

Endpoint in which the error is experienced https://demo.acvts.nist.gov:443/acvp/v1/testSessions/369783/vectorSets/1491314 GET

Expected behavior Expected test vectors to be generated without error.

Additional context Resulting JSON:

[
    {
        "acvVersion": "1.0"
    },
    {
        "error": "Error generating key: Unable to retrieve LMS_TREE for LMS_SHA256_M32_H5_LMOTS_SHA256_N32_W4"
    }
]

Previous attempts on different sessions with similar registration failed with different error message. Eg. for session /acvp/v1/testSessions/369782, vsid 1491311 Error generating key: Unable to retrieve LMS_TREE for LMS_SHA256_M24_H15_LMOTS_SHA256_N24_W4

Other attempts with fewer LMS and LMOTS modes listed in registration succeeded without error.

[livebe01]

Thanks @jbteron, we're looking into this.

[livebe01]

Just an update that we're continuing to troubleshoot this...

[ericeilertson]

Pure speculation, but are you running out of memory or storage when generating the height 25 trees?

[livebe01]

I'm not actively working the issue (others on the team), but not that. The trees are precomputed and stored in our database for retrieval. They are actively working the issue and it looks like we're getting close to having a fix ready.

[livebe01]

Hi @jbteron, we think we have this fixed on our end. Can you try your registration again?

[jbteron]

Running the same registration, it succeeded. However, I have still been seeing the error intermittently for other registrations with other LMS modes. Eg. Test session 377077 vector set 1530532.

[
    {
        "acvVersion": "1.0"
    },
    {
        "error": "Error generating key: Unable to retrieve LMS_TREE for LMS_SHAKE_M32_H25_LMOTS_SHAKE_N32_W8"
    }
]

I've also noticed some other weird behaviour which is possible related this underlying issue. For Test session 377059 vector set 1530480 - the returned testGroups is empty:

[
    {
        "acvVersion": "1.0"
    },
    {
        "algorithm": "LMS",
        "isSample": false,
        "mode": "sigGen",
        "revision": "1.0",
        "testGroups": [],
        "vsId": 1530480
    }
]

[livebe01]

Thanks @jbteron, that's helpful! I'll open a new ticket for the sigGen issue.

[celic]

The tree mentioned is the "worst-case scenario", i.e. the tree that takes the longest amount of time to generate. We pre-generated a set of trees to use for testing so that we can provide immediate vector sets for LMS rather than wait 2-3 days for some of these trees to generate. This one in particular we only have a small handful of trees generated, the fewest of all. This is messing with our "Pools" (where we keep our pre-generated values) due to a configuration error. We'll get that fixed. This will not need a new deployment to address.

If you test everything but that tree, it should work every time on KeyGen.