pingcap / tidb

TiDB is an open-source, cloud-native, distributed, MySQL-Compatible database for elastic scale and real-time analytics. Try AI-powered Chat2Query free at : https://www.pingcap.com/tidb-serverless/
https://pingcap.com
Apache License 2.0
36.98k stars 5.82k forks source link

Queue and S3 functionality #54144

Open thionic opened 3 months ago

thionic commented 3 months ago

Hello! Many applications use a large amount of various complex storage-related state not only in a SQL database, but sometimes in object stores, queues or graph databases. This often leads to the need to use a large number of different tools and greatly increases the maintenance complexity for administrators or DevOps.

It seems to me that this problem could be solved if existing relational databases started adding queue or object storage functionality. This would save a huge amount of time, since there would be no need to separately launch containers with kafka, nats, minio or others. This could save a huge amount of time on maintenance (since it would not have to waste effort on maintaining a huge number of containers with different types of storage). Also, in my opinion, the ability to “subscribe” to updates in certain tables is extremely necessary, not to transfer the task of storing state to other places besides one single database.

I apologize for the abstract formulation and leaving from the established concept of relational databases, but it really seems to me that it would be much simpler if this functionality was available. Thank you!

lance6716 commented 3 months ago

Also, in my opinion, the ability to “subscribe” to updates in certain tables is extremely necessary, not to transfer the task of storing state to other places besides one single database.

This is TiCDC https://docs.pingcap.com/tidb/stable/ticdc-overview#ticdc-%E7%AE%80%E4%BB%8B