Open killme2008 opened 4 months ago
as a table extension similar to ClickHouse
Currently, the CREATE TABLE
clause in GreptimeDB doesn't support switching Engine? From the docs we can only define the columns with a "default engine"/
I'm +1 for a standalone ingester for better scalability, especially in share environment.
What type of enhancement is this?
API improvement, User experience, Other
What does the enhancement do?
Kafka is a popular open-source messaging queue used by many applications for message and data transmission in big-data environments. It is supported as a data source ingestion by numerous databases and data warehouses.
For example, clickhouse supports Kafka engine https://clickhouse.com/docs/en/engines/table-engines/integrations/kafka
Databend has an ingestor project https://github.com/databendcloud/bend-ingest-kafka
Apache Doris supports creating a routine load to import data from kafka https://doris.apache.org/docs/dev/data-operate/import/import-scenes/kafka-load/
As a time series database (TSDB), I believe that greptimedb should also have support for a Kafka connector. This support can be implemented as an independent project or as a table extension similar to ClickHouse.
I am uncertain which approach is preferable, but personally, I lean towards the latter. I have opened this issue to initiate a discussion on the matter.
Implementation challenges
No response