search

nikepan / clickhouse-bulk

Collects many small inserts to ClickHouse and send in big inserts
Apache License 2.0
474 stars 87 forks source link

Respect database specifier in connection string #18

Open joe-bowman opened 4 years ago

joe-bowman commented 4 years ago

So I have two databases, we shall call them default and newdb containing identical tables, and if I connect to clickhouse directly using http://username:password@localhost:8123/default or http://username:password@localhost:8123/newdb I am able to submit queries to the correct database.

However, with clickhouse-bulk, the exact same connection strings as above, inserts to both databases are aggregated into the same DB.

I can (and am now) running a copy of clickhouse-bulk per database, but this seems sub-optimal. Absolute minimum case, clickhouse-bulk should reject queries sent to /newdb if it is only going to insert into a single DB.

nikepan commented 4 years ago

Thank you! It intresting. I will try to make this.

joe-bowman commented 4 years ago

I've tried to work around this in a number of ways.

  1. Specifying the server to clickhouse-bulk pushes to, has zero effect (it always inserts into default) - so running multiple instances doesn't work
  2. Including the database name in the table name (e.g. INSERT INTO newdb.accounts VALUES (...) means the http requests are accepted by clikchouse-bulk (no errors, in logs or response) but nothing is ever flushed to the upstream server
joe-bowman commented 4 years ago

I'll have a look this evening to see if I can get anything working.

romap0 commented 4 years ago

Hello, @nikepan, @joe-bowman. Did you found any solution on this issue?