apla / node-clickhouse

Yandex ClickHouse driver for nodejs
MIT License
216 stars 50 forks source link

Suggest to separate write and read from module clickhouse.js #71

Open yi opened 3 years ago

yi commented 3 years ago

From a real-world production scenario, we've found the current implementation of httpRequest in clickhouse.js is a bit too complicated. It handles both reads (select) and writes (insert). That makes the httpRequest part too hard to be extended and brings some limitations. e.g.

  1. No way to buffer the write response by setting wait_end_of_query. This causes the whole system to fail to handle clickhouse server data-parsing error.
  2. No way to read back response.headers['X-ClickHouse-Summary'] without hacking into the source code of this module.
  3. No way to fetch X-ClickHouse-Progress upgrades when reading back a large chunk of data.

Thus, I'd like to suggest to separate write and read from module clickhouse.js for their significant different behaviours.