Error connecting to cluster from EKS pod

[azuroff]

I'm trying to connect to an MSK cluster from within an EKS pod using this authentication mechanism. It connects to the cluster just fine from my local machine, and it connects from within a Lambda function that's using the same IAM policy as the pod. However, when trying to connect from the pod, I get the following error:

{"level":"ERROR","timestamp":"2022-06-27T21:38:15.881Z","logger":"kafkajs","message":"[SaslAuthenticator-AWS_MSK_IAM] The \"key\" argument must be of type string or an instance of ArrayBuffer, Buffer, TypedArray, DataView, KeyObject, or CryptoKey. Received undefined","broker":"b-2.cluster-broker-url.kafka.us-west-2.amazonaws.com:9098"} {"level":"ERROR","timestamp":"2022-06-27T21:38:15.881Z","logger":"kafkajs","message":"[BrokerPool] The \"key\" argument must be of type string or an instance of ArrayBuffer, Buffer, TypedArray, DataView, KeyObject, or CryptoKey. Received undefined","retryCount":0,"retryTime":271} TypeError [ERR_INVALID_ARG_TYPE]: The "key" argument must be of type string or an instance of ArrayBuffer, Buffer, TypedArray, DataView, KeyObject, or CryptoKey. Received undefined at new NodeError (node:internal/errors:377:5) at prepareSecretKey (node:internal/crypto/keys:580:11) at new Hmac (node:internal/crypto/hash:132:9) at createHmac (node:crypto:165:10) at new Sha256HashConstructor (/app/node_modules/@jm18457/kafkajs-msk-iam-authentication-mechanism/src/Sha256Constructor.js:5:19) at hmac (/app/node_modules/@aws-sdk/signature-v4/dist-cjs/credentialDerivation.js:35:18) at getSigningKey (/app/node_modules/@aws-sdk/signature-v4/dist-cjs/credentialDerivation.js:11:29) at SignatureV4.getSigningKey (/app/node_modules/@aws-sdk/signature-v4/dist-cjs/SignatureV4.js:153:57) at SignatureV4.signString (/app/node_modules/@aws-sdk/signature-v4/dist-cjs/SignatureV4.js:79:49) at process.processTicksAndRejections (node:internal/process/task_queues:95:5) { code: 'ERR_INVALID_ARG_TYPE'

Any ideas where to begin troubleshooting this one?

[ktxed]

@azuroff I would suggest setting-up and running a simple Java-based MSK client to analyze the authentication flow. If you're using aws-msk-iam-auth in debug mode the logging is a lifesaver.

I followed this approach to support role assumption which was necessary in my app (which is not hosted in AWS) and I had to change the sample code provided here to obtain short lived credentials (with a session token).

[Borduhh]

We're getting the same issue trying to connect from a Lambda function.

{
    "errorType": "TypeError",
    "errorMessage": "The \"data\" argument must be of type string or an instance of Buffer, TypedArray, or DataView. Received undefined",
    "code": "ERR_INVALID_ARG_TYPE",
    "stack": [
        "TypeError [ERR_INVALID_ARG_TYPE]: The \"data\" argument must be of type string or an instance of Buffer, TypedArray, or DataView. Received undefined",
        "    at new NodeError (internal/errors.js:322:7)",
        "    at Hmac.update (internal/crypto/hash.js:84:11)",
        "    at Sha256HashConstructor.update (/var/task/index.js:27876:21)",
        "    at hmac (/var/task/index.js:16778:12)",
        "    at getSigningKey (/var/task/index.js:16764:21)",
        "    at processTicksAndRejections (internal/process/task_queues.js:95:5)",
        "    at async SignatureV4.signString (/var/task/index.js:17126:38)",
        "    at async AuthenticationPayloadCreator.create (/var/task/index.js:27960:27)",
        "    at async Object.authenticate (/var/task/index.js:28012:29)",
        "    at async SASLAuthenticator.authenticate (/var/task/index.js:11035:9)"
    ]
}

[Borduhh]

It looks like the problem is originating from here:

const signature = await this.signature.sign(stringToSign, {
      signingDate: new Date(now).toISOString()
    })

in the AuthenticationPayloadCreator.create() function.

[Borduhh]

Looks like the Lambda was not auto-populating the env for the region. Once we added that, it gave a different error.

[jmaver-plume]

@azuroff Can you provide CloudFormation template to test EKS?

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.