search

allwefantasy / spark-binlog

A library for querying Binlog with Apache Spark structure streaming, for Spark SQL , DataFrames and [MLSQL](https://www.mlsql.tech).
Apache License 2.0
154 stars 54 forks source link

Could not find first log file name in binary log index file #27

Open xiaobailong opened 4 years ago

xiaobailong commented 4 years ago

Exception in thread "connect mysql(test, 3306) " java.lang.RuntimeException: com.github.shyiko.mysql.binlog.network.ServerException: Could not find first log file name in binary log index file at org.apache.spark.sql.mlsql.sources.MLSQLBinLogDataSource$$anonfun$org$apache$spark$sql$mlsql$sources$MLSQLBinLogDataSource$$launchBinlogServer$1$1$$anonfun$apply$1.apply(MLSQLBinLogDataSource.scala:155) at org.apache.spark.sql.mlsql.sources.MLSQLBinLogDataSource$$anonfun$org$apache$spark$sql$mlsql$sources$MLSQLBinLogDataSource$$launchBinlogServer$1$1$$anonfun$apply$1.apply(MLSQLBinLogDataSource.scala:152) at org.apache.spark.sql.mlsql.sources.mysql.binlog.BinLogSocketServerInExecutor$$anon$3.onCommunicationFailure(BinLogSocketServerInExecutor.scala:191) at com.github.shyiko.mysql.binlog.BinaryLogClient.listenForEventPackets(BinaryLogClient.java:939) at com.github.shyiko.mysql.binlog.BinaryLogClient.connect(BinaryLogClient.java:579) at org.apache.spark.sql.mlsql.sources.mysql.binlog.BinLogSocketServerInExecutor.org$apache$spark$sql$mlsql$sources$mysql$binlog$BinLogSocketServerInExecutor$$_connectMySQL(BinLogSocketServerInExecutor.scala:261) at org.apache.spark.sql.mlsql.sources.mysql.binlog.BinLogSocketServerInExecutor$$anon$1.run(BinLogSocketServerInExecutor.scala:289) Caused by: com.github.shyiko.mysql.binlog.network.ServerException: Could not find first log file name in binary log index file at com.github.shyiko.mysql.binlog.BinaryLogClient.listenForEventPackets(BinaryLogClient.java:903) ... 3 more

allwefantasy commented 4 years ago

Can you check the following parameters :

bingLogNamePrefix="xxxx"
binlogIndex="xxx"
binlogFileOffset="xxxx"

with show master status; ?

And you can try to remove the checkpoint directory and try again.