Latest Releases

https://github.com/Altinity/clickhouse-sink-connector/releases

Altinity Sink Connector for ClickHouse®

The Altinity Sink Connector moves data automatically from transactional database tables in MySQL and PostgreSQL to ClickHouse for analysis.

Features

Refer Feature Matrix for detailed features.

• Initial data dump and load(MySQL)
• Change data capture of new transactions using Debezium
• Automatic loading into ClickHouse
• Sources: Support for MySQL, PostgreSQL (other databases experimental)
• Target: Support for ClickHouse ReplacingMergeTree
• Able to recover/restart from failures on source or target
• Handle upstream schema changes automatically
• Checksum-based table comparisons
• Scalable to 1000s of tables
• Multiple deployment models
  • Lightweight: single process that transfers from source to target.
  • Kafka: separate source and target processes using Kafka as transport.
• Distribution as Docker container

Getting Started

QuickStart Guide: Lightweight (MySQL)\ QuickStart Guide: Lightweight (PostgreSQL)\ QuickStart Guide: Kafka

Blog Articles

First two are good tutorials on MySQL and PostgreSQL respectively.

• Altinity Sink Connector (MySQL)
• Altinity Sink Connector (PostgreSQL)
• ClickHouse as an analytic extension for MySQL

Reference Documentation

General

• Architecture Overview
• Lightweight Sink Connect CLI
• Mutable Data Handling
• ClickHouse Table Engine Types
• Troubleshooting
• TimeZone and DATETIME/TIMESTAMP
• Replication Start Position
• Logging
• Production Setup
• Adding new tables(Incremental Snapshot)
• Configuration
• State Storage
• Data Type Mapping

Operations

• Monitoring
• Load Testing with Sysbench

Development

• Development
• Testing

Comparison with other technologies

• Comparison

Roadmap

2024 Roadmap

Help

File an issue or contact us on the Altinity public Slack workspace. Use the link on the Slack badge at the top of this page.

Contributing

Contributions to the project are welcome in any form.

• Submit issues documenting feature requests and bugs
• Submit PRs to make changes
• Talk about the project, write blog articles, or give presentations

We recommend that you file an issue before implementing feature additions or major fixes. We are happy to provide guidance and encouragement!

Commercial Support

Altinity is the primary maintainer of the Sink Connector. It is used together with Altinity.Cloud as well as self-managed ClickHouse installations. Altinity.Cloud and is also used in self-managed installations. Altinity offers a range of software and services related to ClickHouse and analytic applications built on ClickHouse.

• Official website - Get a high level overview of Altinity and our offerings.
• Altinity.Cloud - Run ClickHouse in our cloud or yours.
• Altinity Support - Get Enterprise-class support for ClickHouse and Sink Connector.
• Slack - Talk directly with ClickHouse users and Altinity devs.
• Contact us - Contact Altinity with your questions or issues.
• Free consultation - Get a free consultation with a ClickHouse expert today.

Credits

YourKit supports the Altinity Sink Connector and other open source projects with its full-featured Java Profiler. Click the YourKit logo below to learn more.