ERR_OSSL_EVP_DECODE_ERROR when trying to verify authentication

[cz-cs]

I'm having issues with trying to use verifyAuthentication

Server-side uses SvelteKit.

Client-side:

const authenticate = async () => {
  let auth;
  try {
    let challenge = await fetch('/api/challenge');

    auth = await client.authenticate({
      userVerification: 'required',
      challenge: await challenge.json(),
      hints: ['client-device']
    });
  } catch {
    return;
  }

  const verifyRes = await fetch('/api/authenticate', {
    body: JSON.stringify(auth),
    method: 'POST'
  });

  if (verifyRes.status === 400) return;
  if (verifyRes.status === 404) alert('key not found in db');

  let verify: AuthenticationInfo = await verifyRes.json();
  alert(verify.credentialId);
}; 

Server-side:

import { getCredentialKey } from '$lib/server/credentialCollection.js';
import { server } from '@passwordless-id/webauthn';
import type { AuthenticationJSON, CredentialInfo } from '@passwordless-id/webauthn/dist/esm/types.js';

export const POST = async ({ request, cookies }) => {
  const res: AuthenticationJSON = await request.json();
  let doc;
  try {
    doc = await getCredentialKey(res.id);
  } catch {
    cookies.delete('challenge', { path: '/' });
    return new Response(null, { status: 404 });
  }

  const key: CredentialInfo = {
    id: doc!.id,
    publicKey: doc!.publicKey,
    algorithm: doc!.algorithm,
    transports: doc!.transports
  };

  console.log(key);

  let auth;
  try {
    auth = await server.verifyAuthentication(res, key, {
      origin: 'http://localhost:5173',
      challenge: cookies.get('challenge')!,
      userVerified: true
    });
  } catch (e) {
    cookies.delete('challenge', { path: '/' });
    console.error(e); 
    return new Response(null, { status: 400 });
  }

  cookies.delete('challenge', { path: '/' });

  return new Response(JSON.stringify(auth), { status: 200 });
};

Console logs:

DOMException [DataError]: Invalid keyData
    at Object.rsaImportKey (node:internal/crypto/rsa:221:15)
    at SubtleCrypto.importKey (node:internal/crypto/webcrypto:615:10)
    ... 6 lines matching cause stack trace ...
    at async resolve (C:/Users/cz/passkey-test/node_modules/@sveltejs/kit/src/runtime/server/respond.js:462:17)
    at async Module.respond (C:/Users/cz/passkey-test/node_modules/@sveltejs/kit/src/runtime/server/respond.js:338:20) {
  [cause]: Error: error:03000072:digital envelope routines::decode error
      at createPublicKey (node:internal/crypto/keys:619:12)
      at Object.rsaImportKey (node:internal/crypto/rsa:215:21)
      at SubtleCrypto.importKey (node:internal/crypto/webcrypto:615:10)
      at parseCryptoKey (C:\Users\cz\passkey-test\node_modules\@passwordless-id\webauthn\dist\cjs\webauthn.cjs:620:24)
      at verifySignature (C:\Users\cz\passkey-test\node_modules\@passwordless-id\webauthn\dist\cjs\webauthn.cjs:624:25)
      at Object.verifyAuthentication (C:\Users\cz\passkey-test\node_modules\@passwordless-id\webauthn\dist\cjs\webauthn.cjs:567:34)
      at POST (C:/Users/cz/passkey-test/src/routes/api/authenticate/+server.ts:25:47)
      at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
      at async Module.render_endpoint (C:/Users/cz/passkey-test/node_modules/@sveltejs/kit/src/runtime/server/endpoint.js:49:18)
      at async resolve (C:/Users/cz/passkey-test/node_modules/@sveltejs/kit/src/runtime/server/respond.js:462:17) {
    library: 'digital envelope routines',
    reason: 'decode error',
    code: 'ERR_OSSL_EVP_DECODE_ERROR'
  }
}

CredentialInfo for the passkey being used converted back to JSON data from database:

{
  id: '1yihlfmBpWvSXsWOtx0iyS78TyNxr34uTSfxl1SqzOM',
  publicKey: 'MIIBIjANBgkqhkiG9w0BAQsFAAOCAQ8AMIIBCgKCAQEA0ump3K-IfytQtd6U7lBO3sDAfKSc-8OBzK7BGd1tJkxBzvBzn_L4GXFt3PyJgEQzCf39om8FZHjISUeAmCsZmo1TRYjBGwvoPgLgg2qkx6UehWjKf5JVjCfm7EZZxyxCxkn1DeA9rOMEdgmXXRnJBcCAUXGEY_Wi7RZ_rB4ZsgFxHceJdwMNrAuBS3mumxDsjy-XI-k5Yq9xbVFL3SBxV26jvfI0geCF6wnv8Ydhe-PCwu_P4vdgz737FHiM6wP96bNPsMkdoqk8vF6-FE_8VMleolnEtfwE_E2h7E7_aFXlW-SB3unWNJEUDz0xBDCAHcXlXSKyDgEBkh32mgxRdQIDAQAB',
  algorithm: 'RS256',
  transports: [ 'internal' ]
}

I'm using the latest version of the library, Windows 11, with Bun 1.1.29. Registration works okay, and my database stores the credentials correctly.

Any ideas why this doesn't work? Thank you

[dagnelies]

That's strange. It looks like the public key is invalid. Can you also reproduce that error in the playground ? Is it possible that the publicKey value was somehow truncated or altered?

[dagnelies]

In the playground, a button was added to validate the public key, in the lower section

[cz-cs]

Sorry for the delay, I've been caught up in school work, but I can reproduce the error in the playground. I registered for a credential and the browser returns this:

However when I click the button to verify the key, it prompts saying it's not valid. Console logs this

I also tried it on Chrome with same credential, same results. What is interesting is that on my iPhone, on Safari with iOS 18.1, the playground prints that the same public key is valid.

Maybe it's an issue with Windows or my TPM chip?

[dagnelies]

What is interesting is that on my iPhone, on Safari with iOS 18.1, the playground prints that the same public key is valid.

😮 Do I understand you correctly? The same identical public key value is marked INAVLID in Windows/Chrome and VALID in iPhone/Safari?! ...now that's really really weird stuff. After all, it's the same browser code verifying the key.

Or did I understand you incorrectly, and you generated different public keys on Windows (the broken one) and on iPhone (the working one)?

Also, my Windows generates ES256 keys. I wonder why yours generates RS256 keys. 🤔 Is it an older Windows version or so?

[cz-cs]

Yes! I used the same public key and encryption, and it returned different results on both machines

I keep my Windows updated, so I doubt that's the problem here. This is my Windows version: