apla / node-clickhouse

Yandex ClickHouse driver for nodejs
MIT License
216 stars 50 forks source link

Exposing of the "finalized" option #6

Closed tommiv closed 6 years ago

tommiv commented 6 years ago

My problem is to execute a query where INSERT is performed from the aggregated data from another table, for example:

    INSERT INTO temp_regs_${sessionID}
    SELECT *
    FROM (
        SELECT min(date) as day, user_id
        FROM runs
        GROUP BY user_id
    ) as first_regs
    WHERE day BETWEEN ${from} AND ${to}    
    GROUP BY day, user_id

Such query will never end if I'm using a promise interface, because it waits for stream.end() call. I looked into source and found the line https://github.com/apla/node-clickhouse/blob/master/src/clickhouse.js#L289 where request is marked as not finalized. I really want to stick with promises API, because it allows me to use async/await syntax, but now it's not possible for a query above.

For now I came up with a custom wrapper like this:

  function queryFinalized(sql, options) {
    return new Promise((resolve, reject) => {
      const callback = (error, data) => {
        if (error) reject(error);
        else resolve(data);
      };

      const stream = ctx.clickhouse.query(sql, options, callback);

      stream.end();
    });
  }

But maybe it's better to expose finalized flag in req options?

tommiv commented 6 years ago

It turns out that this workaround also requires to force omitFormat: true, because TabSeparated format has been adding automatically.

apla commented 6 years ago

There is no need to expose finalized. From source there is a clear way how to get it fixed: https://github.com/apla/node-clickhouse/blob/master/src/clickhouse.js#L277

tommiv commented 6 years ago

Oh, I got it, so the issue was in SQL detection. Thanks for cooperation, it works well now.