bruin-data / ingestr

ingestr is a CLI tool to copy data between any databases with a single command seamlessly.
https://bruin-data.github.io/ingestr/
MIT License
2.56k stars 62 forks source link

ClickHouse support #1

Open Slach opened 8 months ago

Slach commented 8 months ago

It would be great if your tool will add support for ClickHouse

ChrisMcKee commented 8 months ago

pretty much just passes through to https://dlthub.com/ library so I guess it would need support in https://dlthub.com/docs/dlt-ecosystem/destinations/ first

karakanb commented 8 months ago

Kinda. While dlt destinations would be preferred, I could still add a Clickhouse custom destination without waiting on dlt having it, and then contribute it.

IIRC Clickhouse has a Postgres-compatible API, would that work in the meantime?

e12e commented 8 months ago

IIRC Clickhouse has a Postgres-compatible API, would that work in the meantime?

Have not tried it, but should work according to the documentation:

https://clickhouse.com/docs/en/interfaces/postgresql

And clickhouse can work with a number of sources directly, see eg examples for clickhouse local: https://clickhouse.com/docs/en/operations/utilities/clickhouse-local

ChrisMcKee commented 8 months ago

Kinda. While dlt destinations would be preferred, I could still add a Clickhouse custom destination without waiting on dlt having it, and then contribute it.

IIRC Clickhouse has a Postgres-compatible API, would that work in the meantime?

There's an issue on dlt about using ast to cover a wide range including click house.

Slach commented 8 months ago

ClickHouse support only MySQL/PostgreSQL protocol, but doesn't support MySQL/PostgreSQL SQL dialect.

adrianbr commented 7 months ago

Fiy everyone we are working on it and now we also have custom destinations that enable you to easily add what you want

Slach commented 7 months ago

@adrianbr great news

adrianbr commented 1 month ago

it was added to dlt btw :) forgot to keep you posted

karakanb commented 1 month ago

it is in our roadmap to bring it into ingestr as well, thanks Adrian!