apla / node-clickhouse

Yandex ClickHouse driver for nodejs
MIT License
217 stars 50 forks source link

'end' event is not firing #44

Closed NPCRUS closed 4 years ago

NPCRUS commented 5 years ago

Hello guys. I'm trying to implement some inserting to clickhouse app using you library.

chStream = ch.query('INSERT INTO megas.xovis_events', { format: 'JSONEachRow' })
chStream.on('error', err => {
     console.log(err)
     this.nackAll()
     chStream.end()
})
chStream.on('end', e => {
     this.ackAll()
     console.log('100 batch insertion done ' + new Date())
})
chStream.write(message)
chStream.end()

So after I end stream manually 'end' event has not being fired for some reason. Do you have any ideas? I can provide any additional info you need.

alexeymkh commented 4 years ago

Same problem

cheburakha commented 4 years ago

Hello guys. I'm trying to implement some inserting to clickhouse app using you library.

chStream = ch.query('INSERT INTO megas.xovis_events', { format: 'JSONEachRow' })
chStream.on('error', err => {
     console.log(err)
     this.nackAll()
     chStream.end()
})
chStream.on('end', e => {
     this.ackAll()
     console.log('100 batch insertion done ' + new Date())
})
chStream.write(message)
chStream.end()

So after I end stream manually 'end' event has not being fired for some reason. Do you have any ideas? I can provide any additional info you need.

Switched back my clickhouse-server to 18.14.14 (dockerized) - got my INSERT events again! Driver version is 1.6.0.

nezed commented 4 years ago

@NPCRUS @alexeymkh @cheburakha Can you please provide more info about you environment, CH version and running code?

I'll try to reproduce and investigate this problem

nezed commented 4 years ago

There was two different issues.

  1. First issue described by @NPCRUS was caused by missing .on('data', fn) handler.

    The 'end' event will not be emitted unless the data is completely consumed.

  2. Second issue described by @cheburakha was appeared since ClickHouse v19.3 and fixed in @apla/clickhouse@1.6.3