search

usnistgov / ACVP-Server

A repository tracking releases of NIST's ACVP server. See www.github.com/usnistgov/ACVP for the protocol.
59 stars 20 forks source link

RSA OAEP Decrypt: padding errors #156

Closed smuellerDD closed 2 years ago

smuellerDD commented 3 years ago

environment Demo

testSessionId multiple

vsId multiple

I need to report the following not knowing where the bug lies. All I know is that it was working before (I tested it some months ago when I implemented OAEP support).

When I get an RSA OAEP decryption test vector for the following request, neither OpenSSL nor NSS is able to decrypt it. The error lies in the padding analysis of the input data. By any chance, did something change in OAEP decryption lately? Note, the encryption is successful.

Has anybody seen such issue as well?

        "algorithm":"KTS-IFC",
        "revision":"Sp800-56Br2",
        "function":[
          "partialVal"
        ],
        "iutId":"0123456789abcdef",
        "keyGenerationMethods":[
          "rsakpg1-basic"
        ],
        "modulo":[
          2048,
          3072,
          4096,
          6144,
          8192
        ],
        "fixedPubExp":"010001",
        "scheme":{
          "KTS-OAEP-basic":{
            "kasRole":[
              "initiator",
              "responder"
            ],
            "ktsMethod":{
              "hashAlgs":[
                "SHA2-224",
                "SHA2-256",
                "SHA2-384",
                "SHA2-512",
                "SHA3-224",
                "SHA3-256",
                "SHA3-384",
                "SHA3-512"
              ],
              "supportsNullAssociatedData":true,
              "associatedDataPattern":"uPartyInfo||vPartyInfo",
              "encoding":[
                "concatenation"
              ]
            },
            "l":768
          }
        }
      },
Kritner commented 3 years ago

@smuellerDD can you give some examples of vectors that are failing to decrypt? You should be getting some test vectors that fail to decrypt in the "VAL" test types, but it shouldn't be all of them.

smuellerDD commented 3 years ago

Am Mittwoch, 10. November 2021, 16:45:48 CET schrieb Russ Hammett:

Hi Russ,

@smuellerDD can you give some examples of vectors that are failing to decrypt? You should be getting some test vectors that fail to decrypt in the "VAL" test types, but it shouldn't be all of them.

We are talking about AFT tests. The following is demo test session 211468 / vsID 773769

E.g.:


      {
        "tgId": 2,
        "testType": "AFT",
        "tests": [
          {
            "tcId": 11,
            "iutN": 
"9DC9ECDB0ED4F8C3BDDF2C096315FE5150B2892F14C1B13AD143D03D5BE7085C15A34696342F14CBFAD08334E44D22F6BCBDC426E975D70E6EA249EDE00245C9C1822B9C73233CA63B0909CB962AD372C65BF945A7DC9F9687AEA17CB04850369EF37D1D31E4C59A6112CE973CB8C68D3A1215936975F0929F47C9F2D6183C2652C521378B3486ED6A5A3F58584F23874DC93B64047F3400B4B6D763C97CA46B938FB4477441F36FF84A33C2E98494661FE931E442F78E5386C336DBF0E72F5478FA461E756244685FFC8271DCAD8C84DB776E4A13E8039753000A098542F42ADF474A42671CF14DE937AECCE8D47AC7AF18F56D033103D6EAF8FAEF44841062F7348C1A790F5BBBD5D20E29F949C44870781F3C620B70BA039648A7AE7A779DB9EDE69B4E1B87854EF1F0138ABE7B9BAFF1E93563D548A0210AEB8F5D28DDCF8CDBA6F9BE99735B78E0A0B4A42B939F7CEA0D5E654D72176C8B6BCCE47306F610BE691F6120492D376CCA8DB4BD46C5AF631CBDD2E31E754192DB57A92CFDC7",
            "iutE": "010001",
            "iutP": 
"CE2FB63924C792BA45165562FF0241F13FF8D3FEF3B5C0B0C4B2E2A73C21B714F69F92E360F7E2E28F7ED03A873817153572FA26498B3C6112173006A922391551DD9A161777643A1E1C892F16D3C8F4B88BC9753709F290B6AA5AC43100FE19AEE70E502CCFA1FFE8322F3B1CF20C6CDC6107A83BE7FD5F2DC7E5B9764C6C4E57A5212240A455BF34055D35691F579C63D0119D4F56BFCF0A4EF4828FD5044568985A738E6F3DEEFF56FFFBB707AE2249758F7B92D4E7B219AF927E93551633",
            "iutQ": 
"C3E8E6B4AEC650E5BC102F72D4570B7EBAF218D9DEED8E0EA1CBFCF8B8D785C59C57ADC48245CA209E804A4F40FDBC3E379189784AC91070AC69F978E6EE66065F706F91954210436B63A3EE5D3B779A43E2E61FDEE5350BE8420793883583416FA88AB89700211DB1685D80D5F8B3B3FC81AE6930D6AFEA290792B760C1ABC8E378E711EF741CF14C16F68ED05F127AF9675433FE66B957B97A2CB3F26D34A7F62DDFE873AC1B51B5ACD0EA2D686556BA84249569A1F6C2605ABE0BC5D29E1D",
            "iutD": 
"131270AC2BB74758558F6FA1B4D5D83718F3738C7ACC4CB148D1130F0F9835D4F792943A1C95A244638D2786D19F973DCEA37B5D9EB36D2AA3E9BED8DF4E3704437D09EB6FA94F2FE28BDDA0BB8672A7795D6D9C6345520EDE5897D15BF769A6A880D2DB200840704012876115685A2B80DCE5B9BB3213C18D3A1EA9590275E6C5F368E7EF4E0A8F10DF56AFC580CF510FF0665AD9064D1156C690F3514BDD0DD9B8B1783F8FA72E9E0ACE05DDB6F63EFFD56E7FE22E4D08408E4107C5AF13F3C7CF77E5567236BE9163F9330E979DC656BBF4FB6D01B28034E889E95794E970EDC070E0B9D6D748D7276818530D8A3D0DADADF1DF49935FAF37CE5B5BAAE682D06AA3D2DF6AF4B7FD7E9A1EAACDDF551FB6F2457CF820484ABE14860EA5C7D2D637C855919CC5E12934A65FD48CB5A90BEC9CE85879E58DD084EE0DBEF4E678387D1C7F5191BB346F3C3EF682566AE236BE9FECF86FA10DE9C00E46EE186B16F6F80223004DB6F855ECFD5C30DBDBFB04EF094576E77FB6F867432BBCE3DEB5",
            "serverC": 
"703F99421B8691360B8B6438304F3D833EF5C5D25848A623B7A7F470F5610D75F89251BF773FE0E9A1A475A3D66D7ED0C9C6B6FDB96EC78134CFCAE4541BEFF6CD5AA58FA05A21E2F6C38489C8D741901834EE2C537157F7C1CDFB873F8DDAAF763BB1FC3A8741020335C93CCFB1A13129ED700D7D288901DDD2BFD4B98EC533DAB274BEC616B8E6869F5D4236BFFB58FE670E6A6521C05F5218F594E23FCA622B49FE6950BA5ADCD3138D6E537362EFFB288AC6E5E677BEA2C3766D10C080322EBBEED8BE08712F76AEC80AD2B53FD7A51FC712143C897285C04FC572D51FB28CF6275AB2C36C43B7948D34387B60C868243D0708A3282E09964E153480EFF59C014FAB4858C98C84C5E0113F0FA879C75E309339FE1EA7EEA0C381C6236E752535E68D311BB09EA519419370C8359B68988AF6E543A369BBF578D59BC2D3B285B3880C5F23E9111A6C6A514E9B25D9BD2C9F1B53FE29D770B0B61A77F802F294AC97D27925C8111278D51757316EF5E3F7377C97F83F95C6D33EE9ACBD233E"
          },
...
        "scheme": "KTS-OAEP-basic",
        "kasRole": "responder",
        "keyGenerationMethod": "rsakpg1-basic",
        "modulo": 3072,
        "fixedPubExp": "010001",
        "l": 768,
        "iutId": "0123456789ABCDEF",
        "serverId": "434156536964",
        "ktsConfiguration": {
          "hashAlg": "SHA3-224",
          "associatedDataPattern": "",
          "encoding": "None"
        },
        "keyConfirmationDirection": "",
        "keyConfirmationRole": ""

Ciao
Stephan
Kritner commented 3 years ago

I don't believe that anything has changed around this particular algorithm, but am happy to explore a bit for the test case you posted about above. Is there any sort of pattern you're able to see regarding the number of failing vs passing test cases? Are all failing now? just some?

Here is a test on our side, just to make sure "we're agreeing with our own produced vector", I'll probably be coming back to this over the course of debugging this issue, but just as a baseline:

var iutN = new BitString("9DC9ECDB0ED4F8C3BDDF2C096315FE5150B2892F14C1B13AD143D03D5BE7085C15A34696342F14CBFAD08334E44D22F6BCBDC426E975D70E6EA249EDE00245C9C1822B9C73233CA63B0909CB962AD372C65BF945A7DC9F9687AEA17CB04850369EF37D1D31E4C59A6112CE973CB8C68D3A1215936975F0929F47C9F2D6183C2652C521378B3486ED6A5A3F58584F23874DC93B64047F3400B4B6D763C97CA46B938FB4477441F36FF84A33C2E98494661FE931E442F78E5386C336DBF0E72F5478FA461E756244685FFC8271DCAD8C84DB776E4A13E8039753000A098542F42ADF474A42671CF14DE937AECCE8D47AC7AF18F56D033103D6EAF8FAEF44841062F7348C1A790F5BBBD5D20E29F949C44870781F3C620B70BA039648A7AE7A779DB9EDE69B4E1B87854EF1F0138ABE7B9BAFF1E93563D548A0210AEB8F5D28DDCF8CDBA6F9BE99735B78E0A0B4A42B939F7CEA0D5E654D72176C8B6BCCE47306F610BE691F6120492D376CCA8DB4BD46C5AF631CBDD2E31E754192DB57A92CFDC7");
var iutE = new BitString("010001");
var iutP = new BitString("CE2FB63924C792BA45165562FF0241F13FF8D3FEF3B5C0B0C4B2E2A73C21B714F69F92E360F7E2E28F7ED03A873817153572FA26498B3C6112173006A922391551DD9A161777643A1E1C892F16D3C8F4B88BC9753709F290B6AA5AC43100FE19AEE70E502CCFA1FFE8322F3B1CF20C6CDC6107A83BE7FD5F2DC7E5B9764C6C4E57A5212240A455BF34055D35691F579C63D0119D4F56BFCF0A4EF4828FD5044568985A738E6F3DEEFF56FFFBB707AE2249758F7B92D4E7B219AF927E93551633");
var iutQ = new BitString("C3E8E6B4AEC650E5BC102F72D4570B7EBAF218D9DEED8E0EA1CBFCF8B8D785C59C57ADC48245CA209E804A4F40FDBC3E379189784AC91070AC69F978E6EE66065F706F91954210436B63A3EE5D3B779A43E2E61FDEE5350BE8420793883583416FA88AB89700211DB1685D80D5F8B3B3FC81AE6930D6AFEA290792B760C1ABC8E378E711EF741CF14C16F68ED05F127AF9675433FE66B957B97A2CB3F26D34A7F62DDFE873AC1B51B5ACD0EA2D686556BA84249569A1F6C2605ABE0BC5D29E1D");
var iutD = new BitString("131270AC2BB74758558F6FA1B4D5D83718F3738C7ACC4CB148D1130F0F9835D4F792943A1C95A244638D2786D19F973DCEA37B5D9EB36D2AA3E9BED8DF4E3704437D09EB6FA94F2FE28BDDA0BB8672A7795D6D9C6345520EDE5897D15BF769A6A880D2DB200840704012876115685A2B80DCE5B9BB3213C18D3A1EA9590275E6C5F368E7EF4E0A8F10DF56AFC580CF510FF0665AD9064D1156C690F3514BDD0DD9B8B1783F8FA72E9E0ACE05DDB6F63EFFD56E7FE22E4D08408E4107C5AF13F3C7CF77E5567236BE9163F9330E979DC656BBF4FB6D01B28034E889E95794E970EDC070E0B9D6D748D7276818530D8A3D0DADADF1DF49935FAF37CE5B5BAAE682D06AA3D2DF6AF4B7FD7E9A1EAACDDF551FB6F2457CF820484ABE14860EA5C7D2D637C855919CC5E12934A65FD48CB5A90BEC9CE85879E58DD084EE0DBEF4E678387D1C7F5191BB346F3C3EF682566AE236BE9FECF86FA10DE9C00E46EE186B16F6F80223004DB6F855ECFD5C30DBDBFB04EF094576E77FB6F867432BBCE3DEB5");

var serverC = new BitString("703F99421B8691360B8B6438304F3D833EF5C5D25848A623B7A7F470F5610D75F89251BF773FE0E9A1A475A3D66D7ED0C9C6B6FDB96EC78134CFCAE4541BEFF6CD5AA58FA05A21E2F6C38489C8D741901834EE2C537157F7C1CDFB873F8DDAAF763BB1FC3A8741020335C93CCFB1A13129ED700D7D288901DDD2BFD4B98EC533DAB274BEC616B8E6869F5D4236BFFB58FE670E6A6521C05F5218F594E23FCA622B49FE6950BA5ADCD3138D6E537362EFFB288AC6E5E677BEA2C3766D10C080322EBBEED8BE08712F76AEC80AD2B53FD7A51FC712143C897285C04FC572D51FB28CF6275AB2C36C43B7948D34387B60C868243D0708A3282E09964E153480EFF59C014FAB4858C98C84C5E0113F0FA879C75E309339FE1EA7EEA0C381C6236E752535E68D311BB09EA519419370C8359B68988AF6E543A369BBF578D59BC2D3B285B3880C5F23E9111A6C6A514E9B25D9BD2C9F1B53FE29D770B0B61A77F802F294AC97D27925C8111278D51757316EF5E3F7377C97F83F95C6D33EE9ACBD233E");
var serverK = new BitString("FCC6DDAA3D052160991FDC2EF6C1E9D8D4E15AC47B85B399AFAC6BACA96CB34FBAB6B9231EF9791B3645DE86302ED59A327DBDCE3550BFB5E9FCA3B72058A578A50C5DCB5B1B5592B090153A3B9D9653638C044EE4585C1781143F748B2AAB4B");

var hash = _shaFactory.GetShaInstance(new HashFunction(ModeValues.SHA3, DigestSizes.d224));
_subject = new RsaOaep(hash, new Mgf(hash), new Rsa(new RsaVisitor()), _entropyProviderFactory);

var iutKeyPair = new KeyPair()
{
    PubKey = new PublicKey() { E = iutE.ToPositiveBigInteger(), N = iutN.ToPositiveBigInteger() },
    PrivKey = new PrivateKey()
    {
        D = iutD.ToPositiveBigInteger(),
        P = iutP.ToPositiveBigInteger(),
        Q = iutQ.ToPositiveBigInteger()
    }
};

var result = _subject.Decrypt(iutKeyPair, serverC, null);

Assert.AreEqual(serverK.ToHex(), result.SharedSecretZ.ToHex());

The above test at a minimum passes from our perspective.

Here are the intermediate values of the decrypt operation for that test case. Variable names should be pretty consistent with SP800-56Br2:

{
  "em": "1FD57B8D29EED3DB389AA2C595120ACD6B3A3E8CE800E74B75FEA6F6F29664D2DA4B83580E059A971E51C873FE20467D35F38BDB7D94508CBF45BEFD4D6E6EE30E3545DBCD44309B1913C1440669D13393267892B6141DD9ADA09CD713A81E6D373650ACCBF434C9B96CD7577E1ECF77B398A506D5F3CBD38429FA96940B3238CB46CFD544B24B5A0B19214B2DF315819AD1E687EE7A19EB28A567D8141630E3CD56CCF3E35B283B589C282205626D92BB41C0628CC785BEFBACC563730496A86DB8D842C2AE3ED6B74B7CD0DB2294D860DA9AF2CCBBCAB54A0992226B2F59A51E6D28B1428A6B8207A77C3C250FF0EA94C2AB6A954A67D48C81E5D659233C08D77E152FD6516C85E544EEFF3207972260136A1049C349905F2C31B74D016CE2A891FCB0AA33BAE17A2A8083D0C7F513D256AD72B598BB722400E0418A14F17D606394AACC97A597EE11B3E15AE78140FCB0F7083CAC1095FDCF95B73119B1A788AD80FE0806635EBD9493EDCC4546BD2BDF48752112C61411A94CAD560CA0",
  "EM": "001FD57B8D29EED3DB389AA2C595120ACD6B3A3E8CE800E74B75FEA6F6F29664D2DA4B83580E059A971E51C873FE20467D35F38BDB7D94508CBF45BEFD4D6E6EE30E3545DBCD44309B1913C1440669D13393267892B6141DD9ADA09CD713A81E6D373650ACCBF434C9B96CD7577E1ECF77B398A506D5F3CBD38429FA96940B3238CB46CFD544B24B5A0B19214B2DF315819AD1E687EE7A19EB28A567D8141630E3CD56CCF3E35B283B589C282205626D92BB41C0628CC785BEFBACC563730496A86DB8D842C2AE3ED6B74B7CD0DB2294D860DA9AF2CCBBCAB54A0992226B2F59A51E6D28B1428A6B8207A77C3C250FF0EA94C2AB6A954A67D48C81E5D659233C08D77E152FD6516C85E544EEFF3207972260136A1049C349905F2C31B74D016CE2A891FCB0AA33BAE17A2A8083D0C7F513D256AD72B598BB722400E0418A14F17D606394AACC97A597EE11B3E15AE78140FCB0F7083CAC1095FDCF95B73119B1A788AD80FE0806635EBD9493EDCC4546BD2BDF48752112C61411A94CAD560CA0",
  "HA": "6B4E03423667DBB73B6E15454F0EB1ABD4597F9A1B078E3F5B5A6BC7",
  "Y": "00",
  "maskedMGFSeed": "1FD57B8D29EED3DB389AA2C595120ACD6B3A3E8CE800E74B75FEA6F6",
  "maskedDB": "F29664D2DA4B83580E059A971E51C873FE20467D35F38BDB7D94508CBF45BEFD4D6E6EE30E3545DBCD44309B1913C1440669D13393267892B6141DD9ADA09CD713A81E6D373650ACCBF434C9B96CD7577E1ECF77B398A506D5F3CBD38429FA96940B3238CB46CFD544B24B5A0B19214B2DF315819AD1E687EE7A19EB28A567D8141630E3CD56CCF3E35B283B589C282205626D92BB41C0628CC785BEFBACC563730496A86DB8D842C2AE3ED6B74B7CD0DB2294D860DA9AF2CCBBCAB54A0992226B2F59A51E6D28B1428A6B8207A77C3C250FF0EA94C2AB6A954A67D48C81E5D659233C08D77E152FD6516C85E544EEFF3207972260136A1049C349905F2C31B74D016CE2A891FCB0AA33BAE17A2A8083D0C7F513D256AD72B598BB722400E0418A14F17D606394AACC97A597EE11B3E15AE78140FCB0F7083CAC1095FDCF95B73119B1A788AD80FE0806635EBD9493EDCC4546BD2BDF48752112C61411A94CAD560CA0",
  "mgfSeedMask": "959DA91E5A656C2C01DC87525563C095B9AC41A5C685341A6E1B30FF",
  "mgfSeed": "8A48D293738BBFF739462597C071CA58D2967F292E85D3511BE59609",
  "dbMask": "99D86790EC2C58EF356B8FD2515F79D82A7939E72EF405E426CE3B4BBF45BEFD4D6E6EE30E3545DBCD44309B1913C1440669D13393267892B6141DD9ADA09CD713A81E6D373650ACCBF434C9B96CD7577E1ECF77B398A506D5F3CBD38429FA96940B3238CB46CFD544B24B5A0B19214B2DF315819AD1E687EE7A19EB28A567D8141630E3CD56CCF3E35B283B589C282205626D92BB41C0628CC785BEFBACC563730496A86DB8D842C2AE3ED6B74B7CD0DB2294D860DA9AF2CCBBCAB54A0992226B2F59A51E6D28B1428A6B8207A77C3C250FF0EA94C2AB6A954A67D48C81E5D659233C08D77E152FD6516C85E544EEFF3207972260136A1049C349905F2C31B74D016D1E6E4C568DAF12DA7865F6AE75112E2DC7330C6909302B22DD886B4CE8E6A7BEC7D6DAB7B435EEBEA1ABCF35D174321B72810D393D6C13A57C016C229769BCC90284F04BA51353F1EE2D81A9D651D315DEA7DB0691794ED195059638267CA7EB",
  "DB": "6B4E03423667DBB73B6E15454F0EB1ABD4597F9A1B078E3F5B5A6BC7000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000001FCC6DDAA3D052160991FDC2EF6C1E9D8D4E15AC47B85B399AFAC6BACA96CB34FBAB6B9231EF9791B3645DE86302ED59A327DBDCE3550BFB5E9FCA3B72058A578A50C5DCB5B1B5592B090153A3B9D9653638C044EE4585C1781143F748B2AAB4B",
  "HA2": "6B4E03423667DBB73B6E15454F0EB1ABD4597F9A1B078E3F5B5A6BC7",
  "X": "000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000001FCC6DDAA3D052160991FDC2EF6C1E9D8D4E15AC47B85B399AFAC6BACA96CB34FBAB6B9231EF9791B3645DE86302ED59A327DBDCE3550BFB5E9FCA3B72058A578A50C5DCB5B1B5592B090153A3B9D9653638C044EE4585C1781143F748B2AAB4B",
  "K": "FCC6DDAA3D052160991FDC2EF6C1E9D8D4E15AC47B85B399AFAC6BACA96CB34FBAB6B9231EF9791B3645DE86302ED59A327DBDCE3550BFB5E9FCA3B72058A578A50C5DCB5B1B5592B090153A3B9D9653638C044EE4585C1781143F748B2AAB4B"
}

the loop:

var kStartByte = 0;
for (var i = 0; i < X.BitLength.CeilingDivide(BitString.BITSINBYTE); i++)
{
    if (X[i] == 0x00)
    {
        continue;
    }
    if (X[i] == 0x01)
    {
        kStartByte = i + 1;
        break;
    }

    throw new DecryptionFailedException("X did not have the form PS || 00000001 || K, where PS consists of zero or more consecutive 00 bytes");
}

we're using for step:

d. If X does not have the form PS || 00000001 || K, where PS consists of zero or more consecutive 00 bytes, then DecryptErrorFlag = True.

breaks on an i of 230.

Can you compare to your intermediate values for the same test case and let me know where we differ?

amasino commented 3 years ago

Hi, this test case works ok on our side. I double checked again and I'm enclosing the complete test request vector that fails on our side, the first TC that fails is tgId=4 and tcId=31 (renamed as .txt extension to allow attachment). testvector-request.json.txt 1

Kritner commented 3 years ago

Hey @amasino just trying to confirm "take-aways" from your comment.

Hi, this test case works ok on our side.

Are you saying that the test case @smuellerDD pointed out in comment decrypted fine for you?

But as per the proceeding sentence:

I double checked again and I'm enclosing the complete test request vector that fails on our side, the first TC that fails is tgId=4 and tcId=31 (renamed as .txt extension to allow attachment)...

you are having similar problems with test cases we say should pass, but are not on your side of things? I have not yet taken a look at your attached vector set, I hope to be able to do that today at some point.

amasino commented 3 years ago

When encoding is 'none', we don't have issues. The problem is with encoding="concatenate".

Kritner commented 2 years ago

When encoding is 'none', we don't have issues. The problem is with encoding="concatenate".

@amasino that's just a matter of how your associated data is being constructed, which I'm happy to take onto a different issue to help you resolve. There are numerous other issues that have been closed around the appropriate construction of fixed data and/or associated data for the KAS flavored KDFs and KTS that can be referenced as well. Your issue is most likely a separate issue from @smuellerDD

smuellerDD commented 2 years ago

Am Mittwoch, 10. November 2021, 21:25:41 CET schrieb Russ Hammett:

Hi Russ,

I don't believe that anything has changed around this particular algorithm, but am happy to explore a bit for the test case you posted about above. Is there any sort of pattern you're able to see regarding the number of failing vs passing test cases? Are all failing now? just some?

It seems it is a failure on our side. Thank you very much for your support and apologies for the noise.

Ciao Stephan

Here is a test on our side, just to make sure "we're agreeing with our own produced vector", I'll probably be coming back to this over the course of debugging this issue, but just as a baseline:

var iutN = new BitString(

"9DC9ECDB0ED4F8C3BDDF2C096315FE5150B2892F14C1B13AD143D03D5BE7085C15A3469634
2F14CBFAD08334E44D22F6BCBDC426E975D70E6EA249EDE00245C9C1822B9C73233CA63B0909
CB962AD372C65BF945A7DC9F9687AEA17CB04850369EF37D1D31E4C59A6112CE973CB8C68D3A
1215936975F0929F47C9F2D6183C2652C521378B3486ED6A5A3F58584F23874DC93B64047F34
00B4B6D763C97CA46B938FB4477441F36FF84A33C2E98494661FE931E442F78E5386C336DBF0
E72F5478FA461E756244685FFC8271DCAD8C84DB776E4A13E8039753000A098542F42ADF474A
42671CF14DE937AECCE8D47AC7AF18F56D033103D6EAF8FAEF44841062F7348C1A790F5BBBD5
D20E29F949C44870781F3C620B70BA039648A7AE7A779DB9EDE69B4E1B87854EF1F0138ABE7B
9BAFF1E93563D548A0210AEB8F5D28DDCF8CDBA6F9BE99735B78E0A0B4A42B939F7CEA0D5E65
4D72176C8B6BCCE47306F610BE691F6120492D376CCA8DB4BD46C5AF631CBDD2E31E754192DB
57A92CFDC7"); var iutE = new BitString("010001");
var iutP = new BitString(

"CE2FB63924C792BA45165562FF0241F13FF8D3FEF3B5C0B0C4B2E2A73C21B714F69F92E360
F7E2E28F7ED03A873817153572FA26498B3C6112173006A922391551DD9A161777643A1E1C89
2F16D3C8F4B88BC9753709F290B6AA5AC43100FE19AEE70E502CCFA1FFE8322F3B1CF20C6CDC
6107A83BE7FD5F2DC7E5B9764C6C4E57A5212240A455BF34055D35691F579C63D0119D4F56BF
CF0A4EF4828FD5044568985A738E6F3DEEFF56FFFBB707AE2249758F7B92D4E7B219AF927E93
551633"); var iutQ = new BitString(

"C3E8E6B4AEC650E5BC102F72D4570B7EBAF218D9DEED8E0EA1CBFCF8B8D785C59C57ADC482
45CA209E804A4F40FDBC3E379189784AC91070AC69F978E6EE66065F706F91954210436B63A3
EE5D3B779A43E2E61FDEE5350BE8420793883583416FA88AB89700211DB1685D80D5F8B3B3FC
81AE6930D6AFEA290792B760C1ABC8E378E711EF741CF14C16F68ED05F127AF9675433FE66B9
57B97A2CB3F26D34A7F62DDFE873AC1B51B5ACD0EA2D686556BA84249569A1F6C2605ABE0BC5
D29E1D"); var iutD = new BitString(

"131270AC2BB74758558F6FA1B4D5D83718F3738C7ACC4CB148D1130F0F9835D4F792943A1C
95A244638D2786D19F973DCEA37B5D9EB36D2AA3E9BED8DF4E3704437D09EB6FA94F2FE28BDD
A0BB8672A7795D6D9C6345520EDE5897D15BF769A6A880D2DB200840704012876115685A2B80
DCE5B9BB3213C18D3A1EA9590275E6C5F368E7EF4E0A8F10DF56AFC580CF510FF0665AD9064D
1156C690F3514BDD0DD9B8B1783F8FA72E9E0ACE05DDB6F63EFFD56E7FE22E4D08408E4107C5
AF13F3C7CF77E5567236BE9163F9330E979DC656BBF4FB6D01B28034E889E95794E970EDC070
E0B9D6D748D7276818530D8A3D0DADADF1DF49935FAF37CE5B5BAAE682D06AA3D2DF6AF4B7FD
7E9A1EAACDDF551FB6F2457CF820484ABE14860EA5C7D2D637C855919CC5E12934A65FD48CB5
A90BEC9CE85879E58DD084EE0DBEF4E678387D1C7F5191BB346F3C3EF682566AE236BE9FECF8
6FA10DE9C00E46EE186B16F6F80223004DB6F855ECFD5C30DBDBFB04EF094576E77FB6F86743
2BBCE3DEB5");

var serverC = new BitString(

"703F99421B8691360B8B6438304F3D833EF5C5D25848A623B7A7F470F5610D75F89251BF77
3FE0E9A1A475A3D66D7ED0C9C6B6FDB96EC78134CFCAE4541BEFF6CD5AA58FA05A21E2F6C384
89C8D741901834EE2C537157F7C1CDFB873F8DDAAF763BB1FC3A8741020335C93CCFB1A13129
ED700D7D288901DDD2BFD4B98EC533DAB274BEC616B8E6869F5D4236BFFB58FE670E6A6521C0
5F5218F594E23FCA622B49FE6950BA5ADCD3138D6E537362EFFB288AC6E5E677BEA2C3766D10
C080322EBBEED8BE08712F76AEC80AD2B53FD7A51FC712143C897285C04FC572D51FB28CF627
5AB2C36C43B7948D34387B60C868243D0708A3282E09964E153480EFF59C014FAB4858C98C84
C5E0113F0FA879C75E309339FE1EA7EEA0C381C6236E752535E68D311BB09EA519419370C835
9B68988AF6E543A369BBF578D59BC2D3B285B3880C5F23E9111A6C6A514E9B25D9BD2C9F1B53
FE29D770B0B61A77F802F294AC97D27925C8111278D51757316EF5E3F7377C97F83F95C6D33E
E9ACBD233E"); var serverK =
  new BitString(

"FCC6DDAA3D052160991FDC2EF6C1E9D8D4E15AC47B85B399AFAC6BACA96CB34FBAB6B9231
EF9791B3645DE86302ED59A327DBDCE3550BFB5E9FCA3B72058A578A50C5DCB5B1B5592B0901
53A3B9D9653638C044EE4585C1781143F748B2AAB4B");

var hash = _shaFactory.GetShaInstance(new HashFunction(ModeValues.SHA3,
DigestSizes.d224)); _subject = new RsaOaep(hash, new Mgf(hash), new Rsa(new
RsaVisitor()), _entropyProviderFactory);

var iutKeyPair = new KeyPair()
{
  PubKey = new PublicKey() { E = iutE.ToPositiveBigInteger(), N =
iutN.ToPositiveBigInteger() }, PrivKey = new PrivateKey()
  {
      D = iutD.ToPositiveBigInteger(),
      P = iutP.ToPositiveBigInteger(),
      Q = iutQ.ToPositiveBigInteger()
  }
};

var result = _subject.Decrypt(iutKeyPair, serverC, null);

Assert.AreEqual(serverK.ToHex(), result.SharedSecretZ.ToHex());

The above test at a minimum passes from our perspective.

Here are the intermediate values of the decrypt operation for that test case. Variable names should be pretty consistent with SP800-56Br2:

{
  "em":
"1FD57B8D29EED3DB389AA2C595120ACD6B3A3E8CE800E74B75FEA6F6F29664D2DA4B83580E
059A971E51C873FE20467D35F38BDB7D94508CBF45BEFD4D6E6EE30E3545DBCD44309B1913C1
440669D13393267892B6141DD9ADA09CD713A81E6D373650ACCBF434C9B96CD7577E1ECF77B3
98A506D5F3CBD38429FA96940B3238CB46CFD544B24B5A0B19214B2DF315819AD1E687EE7A19
EB28A567D8141630E3CD56CCF3E35B283B589C282205626D92BB41C0628CC785BEFBACC56373
0496A86DB8D842C2AE3ED6B74B7CD0DB2294D860DA9AF2CCBBCAB54A0992226B2F59A51E6D28
B1428A6B8207A77C3C250FF0EA94C2AB6A954A67D48C81E5D659233C08D77E152FD6516C85E5
44EEFF3207972260136A1049C349905F2C31B74D016CE2A891FCB0AA33BAE17A2A8083D0C7F5
13D256AD72B598BB722400E0418A14F17D606394AACC97A597EE11B3E15AE78140FCB0F7083C
AC1095FDCF95B73119B1A788AD80FE0806635EBD9493EDCC4546BD2BDF48752112C61411A94C
AD560CA0", "EM":
"001FD57B8D29EED3DB389AA2C595120ACD6B3A3E8CE800E74B75FEA6F6F29664D2DA4B8358
0E059A971E51C873FE20467D35F38BDB7D94508CBF45BEFD4D6E6EE30E3545DBCD44309B1913
C1440669D13393267892B6141DD9ADA09CD713A81E6D373650ACCBF434C9B96CD7577E1ECF77
B398A506D5F3CBD38429FA96940B3238CB46CFD544B24B5A0B19214B2DF315819AD1E687EE7A
19EB28A567D8141630E3CD56CCF3E35B283B589C282205626D92BB41C0628CC785BEFBACC563
730496A86DB8D842C2AE3ED6B74B7CD0DB2294D860DA9AF2CCBBCAB54A0992226B2F59A51E6D
28B1428A6B8207A77C3C250FF0EA94C2AB6A954A67D48C81E5D659233C08D77E152FD6516C85
E544EEFF3207972260136A1049C349905F2C31B74D016CE2A891FCB0AA33BAE17A2A8083D0C7
F513D256AD72B598BB722400E0418A14F17D606394AACC97A597EE11B3E15AE78140FCB0F708
3CAC1095FDCF95B73119B1A788AD80FE0806635EBD9493EDCC4546BD2BDF48752112C61411A9
4CAD560CA0", "HA":
"6B4E03423667DBB73B6E15454F0EB1ABD4597F9A1B078E3F5B5A6BC7", "Y": "00",
  "maskedMGFSeed":
"1FD57B8D29EED3DB389AA2C595120ACD6B3A3E8CE800E74B75FEA6F6", "maskedDB":
"F29664D2DA4B83580E059A971E51C873FE20467D35F38BDB7D94508CBF45BEFD4D6E6EE30E
3545DBCD44309B1913C1440669D13393267892B6141DD9ADA09CD713A81E6D373650ACCBF434
C9B96CD7577E1ECF77B398A506D5F3CBD38429FA96940B3238CB46CFD544B24B5A0B19214B2D
F315819AD1E687EE7A19EB28A567D8141630E3CD56CCF3E35B283B589C282205626D92BB41C0
628CC785BEFBACC563730496A86DB8D842C2AE3ED6B74B7CD0DB2294D860DA9AF2CCBBCAB54A
0992226B2F59A51E6D28B1428A6B8207A77C3C250FF0EA94C2AB6A954A67D48C81E5D659233C
08D77E152FD6516C85E544EEFF3207972260136A1049C349905F2C31B74D016CE2A891FCB0AA
33BAE17A2A8083D0C7F513D256AD72B598BB722400E0418A14F17D606394AACC97A597EE11B3
E15AE78140FCB0F7083CAC1095FDCF95B73119B1A788AD80FE0806635EBD9493EDCC4546BD2B
DF48752112C61411A94CAD560CA0", "mgfSeedMask":
"959DA91E5A656C2C01DC87525563C095B9AC41A5C685341A6E1B30FF", "mgfSeed":
"8A48D293738BBFF739462597C071CA58D2967F292E85D3511BE59609", "dbMask":
"99D86790EC2C58EF356B8FD2515F79D82A7939E72EF405E426CE3B4BBF45BEFD4D6E6EE30E
3545DBCD44309B1913C1440669D13393267892B6141DD9ADA09CD713A81E6D373650ACCBF434
C9B96CD7577E1ECF77B398A506D5F3CBD38429FA96940B3238CB46CFD544B24B5A0B19214B2D
F315819AD1E687EE7A19EB28A567D8141630E3CD56CCF3E35B283B589C282205626D92BB41C0
628CC785BEFBACC563730496A86DB8D842C2AE3ED6B74B7CD0DB2294D860DA9AF2CCBBCAB54A
0992226B2F59A51E6D28B1428A6B8207A77C3C250FF0EA94C2AB6A954A67D48C81E5D659233C
08D77E152FD6516C85E544EEFF3207972260136A1049C349905F2C31B74D016D1E6E4C568DAF
12DA7865F6AE75112E2DC7330C6909302B22DD886B4CE8E6A7BEC7D6DAB7B435EEBEA1ABCF35
D174321B72810D393D6C13A57C016C229769BCC90284F04BA51353F1EE2D81A9D651D315DEA7
DB0691794ED195059638267CA7EB", "DB":
"6B4E03423667DBB73B6E15454F0EB1ABD4597F9A1B078E3F5B5A6BC7000000000000000000
0000000000000000000000000000000000000000000000000000000000000000000000000000
0000000000000000000000000000000000000000000000000000000000000000000000000000
0000000000000000000000000000000000000000000000000000000000000000000000000000
0000000000000000000000000000000000000000000000000000000000000000000000000000
0000000000000000000000000000000000000000000000000000000000000000000000000000
0000000000000000000000000000000000000000000000000000000000000001FCC6DDAA3D05
2160991FDC2EF6C1E9D8D4E15AC47B85B399AFAC6BACA96CB34FBAB6B9231EF9791B3645DE86
302ED59A327DBDCE3550BFB5E9FCA3B72058A578A50C5DCB5B1B5592B090153A3B9D9653638C
044EE4585C1781143F748B2AAB4B", "HA2":
"6B4E03423667DBB73B6E15454F0EB1ABD4597F9A1B078E3F5B5A6BC7", "X":
"00000000000000000000000000000000000000000000000000000000000000000000000000
0000000000000000000000000000000000000000000000000000000000000000000000000000
0000000000000000000000000000000000000000000000000000000000000000000000000000
0000000000000000000000000000000000000000000000000000000000000000000000000000
0000000000000000000000000000000000000000000000000000000000000000000000000000
0000000000000000000000000000000000000000000000000000000000000000000000000000
00000001FCC6DDAA3D052160991FDC2EF6C1E9D8D4E15AC47B85B399AFAC6BACA96CB34FBAB6
B9231EF9791B3645DE86302ED59A327DBDCE3550BFB5E9FCA3B72058A578A50C5DCB5B1B5592
B090153A3B9D9653638C044EE4585C1781143F748B2AAB4B", "K":
"FCC6DDAA3D052160991FDC2EF6C1E9D8D4E15AC47B85B399AFAC6BACA96CB34FBAB6B9231E
F9791B3645DE86302ED59A327DBDCE3550BFB5E9FCA3B72058A578A50C5DCB5B1B5592B09015
3A3B9D9653638C044EE4585C1781143F748B2AAB4B" }

the loop in step:

d. If X does not have the form PS || 00000001 || K, where PS consists of zero or more consecutive 00 bytes, then DecryptErrorFlag = True. breaks on an i of 230.

Can you compare to your intermediate values for the same test case and let me know where we differ?

Ciao Stephan