search

pluralsh / plural-artifacts

Artifacts for applications deployable by plural
Apache License 2.0
49 stars 35 forks source link

Onboard TiDB #292

Open troyyyang opened 2 years ago

troyyyang commented 2 years ago

Use Case

TiDB is a hybrid transactional/analytical distributed database (MySQL flavor). I would like to scale with Trino or TiDB, but one of the advantages that TiDB offers over pure analytical processing tools/DBs like Trino/Spark/Snowflake/BQ is having your transactional schemas live in the same database as your analytical/denormalized data, while not compromising performance for either type of workload. Thus reducing ETL/ELT/reverse ETL overhead/maintenance/compute, by truly centralizing all data. I know benchmarks can be extremely hand wavey, but anything within the ballpark of these results is extremely captivating.

image image

Ideas of Implementation

They've implemented an operator, and have pretty good docs.

https://github.com/pingcap/tidb https://github.com/pingcap/tidb-operator https://docs.pingcap.com/tidb-in-kubernetes/stable/deploy-tiflash https://docs.pingcap.com/tidb-in-kubernetes/dev/tidb-toolkit#configure-the-helm-repo https://docs.pingcap.com/tidb-in-kubernetes/stable/deploy-on-azure-aks https://docs.pingcap.com/tidb-in-kubernetes/stable/deploy-on-gcp-gke https://docs.pingcap.com/tidb-in-kubernetes/stable/deploy-on-aws-eks

Additional Info

We're a bit away from needing something beefier than Postgres, but as we approach that, I can take a stab at this if we do go down this route.

Message from the maintainers:

Excited about this feature? Give it a :thumbsup:. We factor engagement into prioritization.

michaeljguarino commented 2 years ago

This is a really interesting project, and haven't heard of it before. We need a decent datawarehouse for our own usecases and it might be a good solve for that

davidspek commented 2 years ago

Not exactly the same since it doesn't support MySQL, but we just finished onboarding YugabyteDB.