ClickHouse / clickhouse-js

Official JS client for ClickHouse DB
https://clickhouse.com
Apache License 2.0
205 stars 25 forks source link

Add RowBinary format for SELECT operations #216

Open slvrtrn opened 8 months ago

slvrtrn commented 8 months ago

Work in progress: #257

If you have any feedback on the following, please don't hesitate to comment.

Streaming will be supported out of the box by design.

Target supported ClickHouse data types with SELECT operations for the initial release; certain types are already completed in the branch:

The types that may be included as well, depending on the format complexity:

Related issues:

cjk commented 5 months ago

@slvrtrn Thanks for working on this! I'm watching this topic closely since it'll be useful for the work I do. My workload also contains Arrays and Maps (to better cope with semi-hierarchical data) and even some Nested types.

Guess I'll wait until Arrays and Maps are supported before jumping in. I also consider moving away from Nested, since it tends to make things more complicated and support for de-/serializing is often lacking / coming last. Often I can use Maps to replace Nested data.

Again, thanks for working on this. I plan at least to support in testing once progress is enough for my usecases. Clickhouse and this Client are both very awesome! :1st_place_medal:

BTW I'm using both INSERTs and SELECTs.

slvrtrn commented 5 months ago

@cjk, thanks for your feedback. I haven't looked into Nested yet, as the core part was to get the numeric types right first, but I believe it will be one of the trickier data types to implement there.

If you could provide a sample dataset for benchmarks or just table DDLs (I will just generate the random data there in the right shape), that would be great.