ML-KEM: server randomly fails

[smuellerDD]

Vsid 2291356 dec: server fails to operate, but 2291244 (same registration) works Vsid 2291243 enc: server fails to operate, but 2291355 (same registration) works

[celic]

I see 2291356 as encapsulation and 2291244 as decapsulation only registrations.

So both of the encapsulation vector sets do not work, but the decapsulation ones both work. Note that for encapsulation we now expect both c and k as outputs. The server is missing a null check on the submitted responses to properly relay this test failure.

[celic]

Do you have an example of a vsId where decapsulation fails? Because decapsulation is a capability and not a mode, it isn't easy to see from a glance from our internal system.

[smuellerDD]

Am Dienstag, 16. April 2024, 17:07:45 MESZ schrieb Chris Celi:

Hi Chris,

Do you have an example of a vsId where decapsulation fails? Because decapsulation is a capability and not a mode, it isn't easy to see from a glance from our internal system.

Apologies, my eyes last night at 11pm were not fresh any more - ML-KEM decap pass all completely. Please ignore my comment.

Ciao Stephan

[smuellerDD]

Status, I tried encap again twice 2 days ago. The server always returned an internal error. That it gave me a verdict with all failures - the shared secret does not match. Note, it was passing before the hotfix.

[celic]

I'm digging into this but I'm a bit at a loss for what's happening. I opened up vsId 2306888 (encapsulation, ML-KEM-1024). The server implementation is consistent with the expected answers. But then it is unclear how your implementation is arriving at a different value. Do we need to open this up to all of the intermediate values too?

This may have "worked" prior to the hotfix because the testing was incomplete in this case. It was essentially just checking that a valid c was returned, i.e. the implicit rejection is not performed. In the current case, we see the implicit rejection is not being performed (it would say so in the validation.json) and instead the shared keys do not match. Separately, the c provided does not match what we expect from c.

I'm at the point of considering a full set of intermediate values to try to see what's happening. It's not like we've changed anything on our implementation, and we still pass the set of regression tests.

It could be around how we are taking values as input from the JSON, similar to ML-DSA, but I can't replicate that issue locally to produce your results.

[smuellerDD]

Am Mittwoch, 1. Mai 2024, 21:45:23 MESZ schrieb Chris Celi:

Hi Chris,

I'm digging into this but I'm a bit at a loss for what's happening. I opened up vsId 2306888 (encapsulation, ML-KEM-1024). The server implementation is consistent with the expected answers. But then it is unclear how your implementation is arriving at a different value. Do we need to open this up to all of the intermediate values too?

This may have "worked" prior to the hotfix because the testing was incomplete in this case. It was essentially just checking that a valid c was returned, i.e. the implicit rejection is not performed. In the current case, we see the implicit rejection is not being performed (it would say so in the validation.json) and instead the shared keys do not match. Separately, the c provided does not match what we expect from c.

I'm at the point of considering a full set of intermediate values to try to see what's happening. It's not like we've changed anything on our implementation, and we still pass the set of regression tests.

It could be around how we are taking values as input from the JSON, similar to ML-DSA, but I can't replicate that issue locally to produce your results.

First of all, thanks for digging. If I can ask you to provide a full set, that would be helpful. I will also check here on my side further.

Ciao Stephan

[smuellerDD]

Am Mittwoch, 1. Mai 2024, 21:45:23 MESZ schrieb Chris Celi:

Hi Chris,

Maybe as a first step, could you also provide the dk in the samples? Otherwise the samples are not helpful.

Ciao Stephan

[celic]

Encapsulation -- ML-KEM-1024.txt

Here is an example. If it helps...

dk = 52D61518179B6947A115E5824FEC0741F07C082A0C46568AAFE9BB0B5761AD94589D495DA4266F10249AF230746D838100C84B8D5C7272E045269224CC00125ABCB82761BF6179919B14719E9975DB4807AFB7ACF8743F8FCCA0DA536FB5CA60E94069012C24D647466659AA56B65FBDC212ACE9556E22B2D429086AA9B494FAC309ACA81D7C4112FA11EC1924977CB108257B5C29391649749BB5002D863F59CA9F7575A60A4C5343C31DA71A229E151770BBB3FD6B062567B6BFAA78895854DAF900856464EB3C7E597764AB6CB0DC23A4283036E0F4137629920B27B1EB13A9141584A4335CBE2C1A72122D26514E2E08AC88541F8FD26F94865E6040568027A2063180DB5097FAC824FFBC6CC531535668ABBAD4358F4756D2A03579B81660BB528CE59570F79F8954BE3A7A057D183284D8BF7FA61C8EC1A658B54A3CA8139094CAD3FC59D366613B55A885C87F0396B44C821FDC903D3314C0E231C94A7B52C1F282B690BFBE0AA4B236564DE90CF6E17571C7A09D692B56598D21A26384C4A87BFAAB131A83491C476406744B949AA5BB273C62B60FD78495E54C0FDB3EDD309A6332879C30494B460BF1A2B8EA747B88F05590DB597BF28554D02636FB3920FC15C0001015880506D2C234063207853DCD784FC5B77F8CB81587A58B0728A435D30ADC6C42722555B306306F4956DAD24A4476CDDAEB8147DC07423072D467447FC6A848F35E8B1369790148099001AA5A57FFE91F3F25887866C2C826BDF1AA09CE2A571ED3ADA1D8AF3CC07FD3053178193514657955D6C9FDD3C114337B999CCD1ED9069D7ABA389C9BEAF92B4B5B69B5F98608488ACCC0931AE35837E87460951925ECB4C43BA1967A0DDA738DF5F6354DE5AF5F533489804BD8187545917F404064032399F6B23D4BD84EB5CB86FD0234F3B37BCB468A1552475F9B924B9CB30B72615ECCA84ACB010C21184CB9A71012AB4B318A1478BD3252C6DC3B9DF04C8E84B15FEAE13AC8FB518FD9244FC1AAD4C10DF1719355280BC0D70DBBB4787148BDD3C86FDC7293D089A588B9252A4276FE271092B3478A26BF8BDB50CCEC416A50BD1DF3AFFAB564E9CBBB68583E8AD4BE5444BC862510E3A0475A275C7B3C174345B9916C47F4728FBFEA4436AAA715193C66137C6DBB31C7B6766F3C7503A2BDD0267F2E475821FB08D233C32F844977A81C143584FCBB935ABB9FDFF164A096047F2498F06013D9190482E82C1C9A0583B06312DC7F0E631028D385FAE6C2892665DE8A0D48681693F81893534C7DCB0609D13F6D132B4F826140F3A14F50AC122A16E73A27C554B3BC82B58D1B8DD9FA833A1B98522C09B6A6028C941E21D259C656244028510D934BEC2A030CE8020839633EB2B17AC5917999C4CB16821205786E787E63E12F01286858001BA40CB10A88B148FC4643390F52FC31BE827DB8A9C46C826143A93226429B0CCCA8A2B33A8D022D992194BB803AD1BC6517A0B39DA14647C815155162DD8697123A082AE8C948C73BAF246F246976527C1CA3CC4DCE372D6E598B4C2B45A6F599792C0F4D76AD0C300FFE47CEE815600A3B2AF3133268B38B5FBC36A6264176C7B9F0A66916E10DFB6C92C27A60F434356C00157D55C75497B95C05A7C5A82F9FD06FF48502D3BB86AA162FFC080AFC4101183B87EED89A1B513C77B0A966C0B1A63A777760B98D4B80A6865FA145065A517BA0890B6ED94EC2251FF1BC302913537BCA4FF6A8B0917AB526494648CC775A1CB775BC69D127515AFCB3D78A276E16ADC1FC1510D0331317C3DA29CD793199EB453741FB1A654AA2BBDC63DF1B8232A9728D6695379CC2CE44346B670888BCA361E432FA9999CA0B802C825D5BD5489570B2F8D5B1F352963F525232817B589AB7BD60A3F73A1F62C318FA416C7F0A3194E50F0581636277B6B06CAB84F058F91C7640161E447664AFE9578051A95323BB737C9C86D81E0A8C66765A95B3EB3CE3C8AC4EC535C3564913DAC6C70C3B19117D23FA9545F4613FD01D3204B40CB541F92A1EEE44C9E3AA52B4058EAA45CE8F36415EA32F1340AD08D43C0411C857DACED05B2893579B0C244A3D1C4ED570B9FCB8ACEF23014C822D2893C946CC73C17967934C0068C83EAEE95CB438707BC1CD64DA2BD6312474DA1BA589C8D49A137E5B38C4501B42F8B51390A7491C13BDB66715198E042B56D3B4C6903BC2563702E333A5E3A85978C73E5ADA6CED3420DDCB9BCAA08C90A16CE6C30A90A214E0EAAD25C741463BB1C7B593408C1F60036498734ECCE704361CAF93C442D9D5C7F046927399104D26B4FAC156A2282EB8667013A10252B550C1184F2B3357DDCB873A4073412AA275CB7B72C54DBB317ABA17180C392342DCC978B2372682CC4E25375946CCAEF6550A15586129A0DD511E61E35E6C7291F5258BB70896CDD690EE3A5FE9286F993B1A968656D54CB5A0D2CF91048243E4CCA124B56AAABE86664060C3B075752CE3EC0B2EF66BB0861BA55124F1667AD8F7290E33BC58B14C79720F84EB7936F4BC461C637AB6CBCF399C66ECB41DB851645240AF7C74262795CD924F3BCA674BC05578A5227E140691D526BE175EE8649767A225F3A1A32F61715079B3128B724782AB5D6CAFB64A57570613CD892065B35147847FC00B1C91E52860B6771DF15E70728EC119145FB322B93732B16CCC0B29AA174469D20475EAA45351E5A304326240219D4A9B77ABB102138A5BB2FA5960D39924A810D863257C1AA1FF659CA82795D4D45C9762040CF6B0669B18692CA04DB5A9365B72445110087AA146329D32B00DED2CB810AC9C420C5AEC82BE20291A7A5C61BF2000E515CA112057F0B199C14A871F99409FE6C970251E8AF1B872593670C27B889C9214B9681E816BF4C687B47148FD4512396C6601A68D222C137DF25C0A533FCD175B2BC640238609127012FAC1648C188FB1D048AC16B62ECA8D96B785295B39BD4396C7A5A6FDD009405628E98A11401655483A2CFDD595F63194849B905828807405D0C0F30E99C4B3877CCCEAC6BA2EA5433AF0AACD32BCC1061EAE7418B3E89CD683A07AA56545E666EF9366D88642E8B8B92AA159805432F1FB2F7A3466934A3D6FE2B054AB4222F721BEAC98F9E62C7D3501DC571FB10355FAC95DA1E3109A2B8E6653A2E1142BFB66C811A831D6723CDE82B5D53618EFDC5BFB752157FB83B40495248AA9B1728AC2020A52753166867B53E17C453A4B7E5B95876CA68D4C0DDE67CDA2F816965C2EE5BAC7DD4A004DA4956CFAA52CE0AF67374D57979ED3FC66AC3CB1E0D795F82364C3825EAAC8CB273AB8392B68EB255FDB27C28D2664B530508E16A1121468B3594BCF427134206C9CC79AD1485F40065590F5CBE3882FBFDB6249F41B6F785A68E0C0BC689F0DB737B53C5DBEA321D117A7A6A6BE5B678E3A4A52EC273840B1CFC2932F5EE645067205E77925C24B1CD8E8A98742AD93A406B3F04815AA94CEE878D1B63A7C7B8DCE5B70B90110D481CAC396541BA483729918CF716645ACB465D4A2493A8F1F9365729912A041BF63F08D862122C5BB25DFCA93747C4E44E4A900FCBA68E245520BB30CF2186DB5C0D9749855141324CBAB0CDA4F96877C976B26A09B8FDA3471741C4B883A77A765728DF5413EF634E083003FD30F11D98619857439548C72FC3C7818142B2797B70CB01A31813A2B8F4A4094F7B3BDD1F39A5E119FDB06CEA8B6C0A02CB06AF711DA3B7B91F798E9E28C09893F30082CA7571C63DC734AA4B710246AE57B5FED09BCB0D21C5852AD49278D645BBD49B20A72B88B290ABE5187815D86AC2D8323019366B80A8EA444CB56C189BDC47CF3946630775FDD641C40652CC0922ACE991C6C4B1D0F9A04546BC71EFA51AC358B2AE27F000B5ABA239F449C8FDB10B59B01A6C52308AD14C4D1C2910F8055FEC45A89B8AF9D9B23F146324425837830CB4339AFB2D831CC56A668F93A34AC500CBA8057CBAF66462767B2C703E86E62E7C2D9D832AA71213C78965CD49546115DBB2776C44600DAB1BA29470A664440CB778B5ED8106B1172E7B18F5D63791731B5DE2A9EE64CC4F32184F3AB2AB4C045AC6A4D1044BD2237C592D3188CE6AC8C80105B04516D90761002794BE3A05E5602E0A33F1839AD7BA3CDCE4CA3A398C2A63940C7CB0102934B470729847541104725AB0B5C11F8203E610C2B4A85A5679309728FB5241EA754AEE47C33ECE7063909578BD6C67E960BB5C11FC57068E0BC79E5D20E6B940C4DE40EDA3055730BA57FAC7DEC029D3042827E02691662ABBD882D45384759E13FB6F5190ECEED1A78639A837A6C10298EA4E8FFFA1EDB60600B5C50936B711B27E80CDDBCE0C5FABA2D6706C63B425821E983C0B81CA3F51CFDCC5CD8EE76E56D2C9BA058DF901F99ED7E1BF227FA63641F66D19FFE8D2004F70025CEDBCCC0F4839D25

[smuellerDD]

This is a big plamface! I did not know what to do with the m value. So, I ignored it. But it is the “random” message in lieu of the output of the RNG! After applying the m value as RNG output, it passes.

ML-KEM fully works now.