slvrtrn
commented
1 year ago Interesting, thanks for reporting.
Adding a
await result.json()to the example improves performance of keep alive true
I think that's because the underlying socket is released once the stream is consumed or destroyed and is returned back to the pool.
You also could try to reduce the number of max_connections in config.xml, cause by default, it is set to 1024.
For example:
config.xml
<max_connections>10</max_connections>index.ts
import { ClickHouseClient, createClient } from "@clickhouse/client";
import pLimit from "p-limit";
const host = "http://localhost:8123";
const username = "default";
const password = undefined;
const requests = 1000;
const workers = 20;
async function main() {
await runPerformanceTest(true);
await runPerformanceTest(false);
}
async function runPerformanceTest(keep_alive: boolean) {
const client = await createClickHouseClient(keep_alive);
const averageResponseTime = await measureAverageResponseTime(client);
console.log(
`Average response time with keep alive ${keep_alive}: ${averageResponseTime}ms`
);
await client.close();
}
async function measureAverageResponseTime(client: ClickHouseClient) {
const limit = pLimit(workers);
const queries = Array.from({ length: requests }, () =>
limit(async () => {
const start = process.hrtime();
const rs = await client.query({
query: "SELECT * FROM system.tables", // around 30 tables here
format: "JSONEachRow",
});
await rs.json();
const responseTime = process.hrtime(start)[1] / 1000000;
return responseTime;
})
);
const responseTimes = await Promise.all(queries);
return (
responseTimes.reduce((a: any, b: any) => a + b, 0) / responseTimes.length
);
}
async function createClickHouseClient(keep_alive: boolean) {
return createClient({
host,
username,
password,
keep_alive: {
enabled: keep_alive,
},
compression: {
response: false,
request: false,
},
});
}
main().catch((err) => {
console.error(err);
process.exit(1);
});I ran a few quick tests on my machine (Linux, Node v16.17.0)
With default max_connections value (1024), KeepAlive is significantly faster, but overall it's pretty slow in both scenarios:
➜ clickhouse-js-performance git:(main) ✗ yarn start
[INFO] 18:14:59 ts-node-dev ver. 2.0.0 (using ts-node ver. 10.9.1, typescript ver. 5.1.6)
Average response time with keep alive true: 229.34405645900026ms
Average response time with keep alive false: 281.2647160240004msWith max_connections set to 10:
[INFO] 18:55:42 ts-node-dev ver. 2.0.0 (using ts-node ver. 10.9.1, typescript ver. 5.1.6)
Average response time with keep alive true: 83.33827950300005ms
Average response time with keep alive false: 156.89417462500006mshere's a simplified version of the client in the nutshell that runs similar code:
import Http from "http";
import pLimit from "p-limit";
import Stream from "stream";
const requests = 1000;
const workers = 20;
async function main() {
await runPerformanceTest(true);
await runPerformanceTest(false);
}
async function runPerformanceTest(keepAlive: boolean) {
const agent = new Http.Agent({
keepAlive,
});
const averageResponseTime = await measureAverageResponseTime(agent);
console.log(
`Average response time with keep alive ${keepAlive}: ${averageResponseTime}ms`
);
agent.destroy();
}
async function measureAverageResponseTime(agent: Http.Agent) {
const limit = pLimit(workers);
const headers = {
Authorization: `Basic ${Buffer.from("default:").toString("base64")}`,
};
const queries: Promise<number>[] = Array.from({ length: requests }, (_, i) =>
limit(async () => {
const start = process.hrtime();
return new Promise((resolve, reject) => {
const bodyStream = Stream.Readable.from([
"SELECT * FROM system.tables FORMAT JSONEachRow",
]);
const request = Http.request("http://localhost:8123", {
method: "POST",
agent,
headers,
});
request.on("response", async (response) => {
if (response.statusCode !== 200) {
console.error(`Rejected with ${response.statusCode}`);
reject(new Error("request failed"));
}
await getAsText(response);
const responseTime = process.hrtime(start)[1] / 1000000;
resolve(responseTime);
});
Stream.pipeline(bodyStream, request, () => {});
});
})
);
const responseTimes = await Promise.all(queries);
return (
responseTimes.reduce((a: any, b: any) => a + b, 0) / responseTimes.length
);
}
main().catch((err) => {
console.error(err);
process.exit(1);
});
async function getAsText(stream: Stream.Readable): Promise<string> {
let result = "";
const textDecoder = new TextDecoder();
for await (const chunk of stream) {
result += textDecoder.decode(chunk, { stream: true });
}
// flush
result += textDecoder.decode();
return result;
}results are in line with what we see when using ClickHouseClient itself:
[INFO] 18:54:32 ts-node-dev ver. 2.0.0 (using ts-node ver. 10.9.1, typescript ver. 5.1.6)
Average response time with keep alive true: 81.46500756599997ms
Average response time with keep alive false: 154.66927978100017ms
kiliangrashoff
den-crane
Describe the bug
Using keep-alive with clickhouse-js degrades performance of concurrent queries compared to disabling keep-alive.
Steps to reproduce
Numbers vary quite a bit, but keep alive false is consistently faster. Adding a
await result.json()to the example improves performance of keep alive true, but this is not what we observe with real workloads.With a real workload, performance of our api degrades from 95ms to 120ms average response time when enabling keep-alive. We configured keep_alive with the following settings and a keep_alive_timeout of 60s on the server, this does not fix the issue.
Expected behaviour
Enabling keep-alive should result in better performance.
Executing a
SELECT 1query twice with curl to time the HTTP transaction with an open TCP connection results in 0.6ms response time for the second request, I would expect the JS client to be able to query an open connection with similar response time:Code example
Note: example depends on
p-limitversion3.1.0for concurrency control, easiest way to run it is to clone the repo I linked.Configuration
Environment
ClickHouse server