High Availability - Reads

bluestreak01 commented 3 years ago

Summary

Automated asynchronous table replication from primary to one or more secondary copies. Replication will work by way of distributing WAL (write-ahead-log) segments via S3. Replica tables will maintain its state by downloading and applying WAL segments.

This implementation depends on:

[x] #2
[x] #1

Expected outcome

replication is configured at table level
replication is to work against AWS S3 API, including on-prem platforms, such as https://min.io/
replication is to work across multiple AWS availability zones
WAL segment application can be time-boxed to support Point-In-Time recovery
replicated table is to support all data and metadata changes, including CREATE TABLE, DROP TABLE and RENAME TABLE SQLs

Timelines

High Availability has just been released for QuestDB Enterprise, and is in the process of being integrated into QuestDB Cloud. It will be released into open source later (tbd).

Menniti commented 3 years ago

Hello @bluestreak01,

Thanks for sharing the information. Do you know if clustering features will be available open-source? Where will be possible to operate 24/07.

Thanks!

bluestreak01 commented 3 years ago

that's the plan 👍

romanzdk commented 1 year ago

Hi all, is there any status update on high availability?

questdb / roadmap