Tests SASL_SCRAM_SHA512 (SASL over TLS)

[blezoray]

Hello,

I'm testing your test_sasl_auth.js script with a SCRAM_SHA512 user.

But I have this error:

$ k6 run -v k6-sasl.js 
DEBU[0000] Logger format: TEXT                          
DEBU[0000] k6 version: v0.39.0 ((devel), go1.18.4, linux/amd64) 

          /\      |‾‾| /‾‾/   /‾‾/   
     /\  /  \     |  |/  /   /  /    
    /  \/    \    |     (   /   ‾‾\  
   /          \   |  |\  \ |  (‾)  | 
  / __________ \  |__| \__\ \_____/ .io

DEBU[0000] Resolving and reading test 'k6-sasl.js'...   
DEBU[0000] Loading...                                    moduleSpecifier="file:///k6-scripts/k6-sasl.js" originalModuleSpecifier=k6-sasl.js
DEBU[0000] 'k6-sasl.js' resolved to 'file:///k6-scripts/k6-sasl.js' and successfully loaded 3996 bytes! 
DEBU[0000] Gathering k6 runtime options...              
DEBU[0000] Initializing k6 runner for 'k6-sasl.js' (file:///k6-scripts/k6-sasl.js)... 
DEBU[0000] Detecting test type for...                    test_path="file:///k6-scripts/k6-sasl.js"
DEBU[0000] Trying to load as a JS test...                test_path="file:///k6-scripts/k6-sasl.js"
DEBU[0001] Babel: Transformed                            t=675.613151ms
ERRO[0001] ReferenceError: SASL_SCRAM_SHA512 is not defined
    at file:///k6-scripts/k6-sasl.js:38:15(58)  hint="script exception"

I use your docker image: mostafamoradian/xk6-kafka:latest

Any explanation ?

The script:

/*

This is a k6 test script that imports the xk6-kafka and
tests Kafka with a 200 JSON messages per iteration. It
also uses SASL authentication.

*/

import { check } from "k6";
import { Writer, Reader, Connection, SASL_PLAIN, TLS_1_2 } from "k6/x/kafka"; // import kafka extension

export const options = {
    // This is used for testing purposes. For real-world use, you should use your own options:
    // https://k6.io/docs/using-k6/k6-options/
    scenarios: {
        sasl_auth: {
            executor: "constant-vus",
            vus: 1,
            duration: "10s",
            gracefulStop: "1s",
        },
    },
};

const brokers = ["my-cluster-kafka-bootstrap.diod-mpms-kafka-test.svc:9092"];
const topic = "bench1";

// SASL config is optional
const saslConfig = {
    username: "benchuser",
    password: "XXXXXXXX",
    // Possible values for the algorithm is:
    // NONE (default)
    // SASL_PLAIN
    // SASL_SCRAM_SHA256
    // SASL_SCRAM_SHA512
    // SASL_SSL (must enable TLS)
    algorithm: SASL_SCRAM_SHA512,
};

// TLS config is optional
const tlsConfig = {
    // Enable/disable TLS (default: false)
    enableTls: false,
    // Skip TLS verification if the certificate is invalid or self-signed (default: false)
    insecureSkipTlsVerify: false,
    // Possible values:
    // TLS_1_0
    // TLS_1_1
    // TLS_1_2 (default)
    // TLS_1_3
    minVersion: TLS_1_2,

    // Only needed if you have a custom or self-signed certificate and keys
    // clientCertPem: "/k6-scripts/benchuser.user.crt",
    // clientKeyPem: "/k6-scripts/benchuser.user.key",
    // serverCaPem: "/k6-scripts/benchuser.ca.crt",
};

const offset = 0;
// partition and groupID are mutually exclusive
const partition = 0;
const numPartitions = 1;
const replicationFactor = 1;
const groupID = "benchusergroup";

const writer = new Writer({
    brokers: brokers,
    topic: topic,
    sasl: saslConfig,
    tls: tlsConfig,
});
const reader = new Reader({
    brokers: brokers,
    topic: topic,
    // partition: partition,
    groupID: groupID,
    offset: offset,
    sasl: saslConfig,
    tls: tlsConfig,
});
const connection = new Connection({
    address: brokers[0],
    sasl: saslConfig,
    tls: tlsConfig,
});

if (__VU == 0) {
    connection.createTopic({
        topic: topic,
        numPartitions: numPartitions,
        replicationFactor: replicationFactor,
    });
    console.log("Existing topics: ", connection.listTopics(saslConfig, tlsConfig));
}

export default function () {
    for (let index = 0; index < 100; index++) {
        let messages = [
            {
                key: JSON.stringify({
                    correlationId: "test-id-abc-" + index,
                }),
                value: JSON.stringify({
                    name: "xk6-kafka",
                    version: "0.2.1",
                    author: "Mostafa Moradian",
                    description:
                        "k6 extension to load test Apache Kafka with support for Avro messages",
                    index: index,
                }),
            },
            {
                key: JSON.stringify({
                    correlationId: "test-id-def-" + index,
                }),
                value: JSON.stringify({
                    name: "xk6-kafka",
                    version: "0.2.1",
                    author: "Mostafa Moradian",
                    description:
                        "k6 extension to load test Apache Kafka with support for Avro messages",
                    index: index,
                }),
            },
        ];

        writer.produce({ messages: messages });
    }

    // Read 10 messages only
    let messages = reader.consume({ limit: 10 });
    check(messages, {
        "10 messages returned": (msgs) => msgs.length == 10,
    });
}

export function teardown(data) {
    if (__VU == 0) {
        // Delete the topic
        connection.deleteTopic(topic);
    }
    writer.close();
    reader.close();
    connection.close();
}

Rgds.

[mostafa]

Hey @blezoray,

You need to import the constant at the top:

// import kafka extension and constants
import { Writer, Reader, Connection, SASL_SCRAM_SHA512, TLS_1_2 } from "k6/x/kafka";

I'll close this ticket, but feel free to re-open it if you have other questions.

[blezoray]

With this import, it's better:

import { Writer, Reader, Connection, SASL_PLAIN, SASL_SCRAM_SHA512, TLS_1_2 } from "k6/x/kafka"; // import kafka extension

[blezoray]

Thanks

[blezoray]

Now, I'm trying to test a listerner with SASL over TLS. I configure the tlsConfig.serverCaPem with the path of my CA file.

// TLS config is optional
const tlsConfig = {
    // Enable/disable TLS (default: false)
    enableTls: true,
    // Skip TLS verification if the certificate is invalid or self-signed (default: false)
    insecureSkipTlsVerify: false,
    // Possible values:
    // TLS_1_0
    // TLS_1_1
    // TLS_1_2 (default)
    // TLS_1_3
    minVersion: TLS_1_2,

    // Only needed if you have a custom or self-signed certificate and keys
    // clientCertPem: "/k6-scripts/benchuser.user.crt",
    // clientKeyPem: "/k6-scripts/benchuser.user.key",
    serverCaPem: "/k6-scripts/benchuser.ca.crt",
};

But I have this error:

ERRO[0001] Cannot process TLS config                     error="File not found: , OriginalError: %!w(*fs.PathError=&{stat  2})"
ERRO[0001] Cannot process TLS config                     error="File not found: , OriginalError: %!w(*fs.PathError=&{stat  2})"
ERRO[0001] Cannot process TLS config                     error="File not found: , OriginalError: %!w(*fs.PathError=&{stat  2})"
ERRO[0001] Failed to create dialer., OriginalError: %!w(*fmt.wrapError=&{could not successfully authenticate to my-cluster-kafka-bootstrap.diod-mpms-kafka-test.svc:9093 with SASL: SASL handshake failed: EOF 0xc0015d8e00})  error="Failed to create dialer., OriginalError: %!w(*fmt.wrapError=&{could not successfully authenticate to my-cluster-kafka-bootstrap.diod-mpms-kafka-test.svc:9093 with SASL: SASL handshake failed: EOF 0xc0015d8e00})"
ERRO[0001] Failed to create dialer., OriginalError: %!w(*fmt.wrapError=&{could not successfully authenticate to my-cluster-kafka-bootstrap.diod-mpms-kafka-test.svc:9093 with SASL: SASL handshake failed: EOF 0xc0015d8e00})
    at file:///tmp/k6-sasl-tls.js:84:19(124)  hint="script exception"

It seems to open 3 files ???

It works fine when I remove the serverCaPem and I set insecureSkipTlsVerify to true.

Any reason ?

[mostafa]

@blezoray You need to provide all three files to be able to test over TLS. Also, if you pass insecureSkipTlsVerify, it'll completely bypass validation.

[blezoray]

But if my listerner does SASL authentication over TLS, I have only the CA. With CA, user cert & key, it is TLS authentication and not SASL authentication.

[mostafa]

@blezoray What do you mean? Can you elaborate?

[mostafa]

I suppose @oscar067 has managed to make SASL/SSL auth work.

[blezoray]

Here, you have an explanation of the difference between TLS 1 way and TLS 2 way. https://www.cloudflare.com/learning/access-management/what-is-mutual-tls/

In your case, when you do SASL over TLS, you establish a TLS 1way session to the server and then SASL challenges the client to be authenticated with its user/password.

When you do TLS 2 ways, your client is authenticated inside the TLS session with its private key/cert and the server doesn't need to do SASL challenge.

Is it clear ?

[oscar067]

Hi

I was able to make work the SSL TLS 2 way, as @mostafa describe, but is true I did not tried SASL over TLS

export const writerCommsHub = new Writer({
  // WriterConfig object
  brokers: bootstrap,
  topic: kafkaTopic,
  tls: {
    enableTls: true,
    insecureSkipTlsVerify: false,
    clientCertPem: "/certs/cert.pem",
    clientKeyPem:  "/certs/server.key",
    serverCaPem:   "/certs/Corporate_Root_CA_G3_.cer",
  },
});

[mostafa]

@oscar067 Thanks for letting us know. @blezoray Then this is something I need to investigate more.

[blezoray]

Hi @mostafa , I found this video which explains the difference between TLS and SASL over TLS. https://www.youtube.com/watch?v=_PmEs8xEz8g

Rgds.

[mostafa]

@blezoray Awesome! Thanks for the pointer.

I'll try to see if I can fix it. In the meantime, I'd be happy to see contributions. SASL and TLS are handled in the auth.go file.

[mostafa]

@blezoray Created #169 to fix this issue.

[mostafa]

@blezoray Fixed in #170. Feel free to reopen the issue if it problem persists.

[blezoray]

Hi,

Sorry for the delay to answer. I tested SASL auth, SASL auth over TLS, and TLS auth. All works fine. Thanks a lot.

Rgds.