EPROTO error

[aimfeld]

When running INSERT queries using clickhouse-js, we often get this error (when running the same queries using Datagrip, this error does not occur):

Error: write EPROTO 00182F3DD67F0000:error:0A000438:SSL routines:ssl3_read_bytes:tlsv1 alert internal error:../deps/openssl/openssl/ssl/record/rec_layer_s3.c:1590:SSL alert number 80

    at WriteWrap.onWriteComplete [as oncomplete] (node:internal/stream_base_commons:94:16) {
  errno: -71,
  code: 'EPROTO',
  syscall: 'write'
}

We checked the clickhouse-server.err.log, and we see corresponding errors like this:

2024.05.13 09:04:00.372128 [ 19143 ] {} <Error> ServerErrorHandler: Code: 210. DB::NetException: SSL Exception: error:0A000115:SSL routines::session id context uninitialized, while reading from socket (peer: [************]:41224, local: [************]:8443). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below):

0. DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000c9a449b
1. DB::NetException::NetException<String, String, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type, std::type_identity<String>::type>, String&&, String&&, String&&) @ 0x000000000caec03e
...

It could be a network/SSL configuration problem in our clickhouse cluster, but maybe the error is related to clickhouse js somehow?

We establish the connection like this, using a .pem file:

const clickHouseClient = createClient({
    host: options.clickhouseHost,
    database: options.clickhouseDatabase,
    username: options.clickhouseUser,
    password: options.clickhousePassword,
    application: 'clickhouse-seeder',
    log: { level: ClickHouseLogLevel.WARN },
    tls: options.clickhouseCert ? { ca_cert: fs.readFileSync(options.clickhouseCert) } : undefined
});

• Client version: 1.0.1
• ClickHouse Server version: 24.4.1.2088

[slvrtrn]

It could be a certificate issue.

FWIW, the basic/mutual TLS support is tested with crt/key files: https://github.com/ClickHouse/clickhouse-js/blob/main/packages/client-node/__tests__/tls/tls.test.ts

The certificate buffers are passed to the agent as-is: https://github.com/ClickHouse/clickhouse-js/blob/main/packages/client-node/src/connection/node_https_connection.ts#L15-L17, no extra magic there.

[aimfeld]

After disabling SSL verification in our cluster, the error has been resolved. May be related to https://stackoverflow.com/questions/38658473/ssl-handshake-error-session-id-context-uninitialized

[slvrtrn]

For the future purposes (an excerpt from the Slack thread related to this issue):

• There is an LB between the client app and CH
• As the LB handles the SSL, it might not be necessary to have it in strict mode on the nodes
• Setting verificationMode to none fixed the issue (see the overview)
• It could be that due to the KeepAlive connections being established with an LB and not ClickHouse itself, sometimes a particular App <-> LB connection got a mismatching internal LB <-> CH one, and that was causing the error (not 100% sure).