search

allwefantasy / spark-binlog

A library for querying Binlog with Apache Spark structure streaming, for Spark SQL , DataFrames and [MLSQL](https://www.mlsql.tech).
Apache License 2.0
154 stars 54 forks source link

解决批次内全量拉取binlog更新量导致socket异常问题 #5

Closed bebee4java closed 4 years ago

bebee4java commented 4 years ago

修改socket传输协议为[head length data end]增加启止位持续交付，而不是全量放入内存，并将socket数据封装成Iterator返回

allwefantasy commented 4 years ago

非常感谢！