pingcap / community

TiDB community content
Apache License 2.0
260 stars 151 forks source link

Incubating Feature: steam bulk insert in IoT scenario #408

Open fanweixiao opened 3 years ago

fanweixiao commented 3 years ago

Feature Incubating Request

In the IoT industry, integrated with YoMo to reduce the pressure and the cost when storing data to the cloud.

Describe the feature you want to incubate:

We are introducing YoMo a streaming serverless framework to the IoT industry. In this scenario, IoT devices/sensors always generate data at high frequency, when hundreds of these data sources connected, the writing pressure makes our customers put Kafka before DB, as we know, Cloud Kafka is expensive, result in bad ROI for our customers. By YoMo, developers can implement a bulk insert to the backend database whenever every 10,000 data arrived or every 30s like yomo-sink-faunadb-example

BTW, YoMo was built atop IETF QUIC Protocol, and designed as an event-first system, as gRPC is busy implementing over HTTP/3, and here, someday these two products can communicate over QUIC, cause in IoT, QUIC shows lots of advantages.

c4pt0r commented 3 years ago

Thank you! And really appreciate you also post a FaunaDB example, that would be helpful.

fanweixiao commented 3 years ago

@c4pt0r We implement one at https://github.com/yomorun/yomo-sink-tidb-example, please check