Running spark connector on DataBricks 13.1 (powered by Spark 3.4.0) will fail because of the below exception. The version of spark connector is 1.1.0.
How to solve it
DBR 13.1 is powered by Spark 3.4.0, but it also includes an additional improvement https://issues.apache.org/jira/browse/SPARK-42968. The improvement introduces a default method useCommitCoordinator in interface StreamingWrite. Because StarRocksWrite implements both interface BatchWrite and StreamingWrite which both have default method useCommitCoordinator on DBR 13.1, so it will leads default method conflict as the exception says. We could override the method in StarRocksWrite to solve it. see https://www.geeksforgeeks.org/resolving-conflicts-during-multiple-inheritance-in-java/.
This improvement is introduced since Spark 3.5, but DBR introduce it in advance, and that's why the connector can run on community Spark 3.4, but failed on DBR 13.1. It seems DBR is not completely compatible with the community Spark.
What type of PR is this：
Which issues of this PR fixes ：
Fixes #
Problem Summary(Required) ：
What's the problem
Running spark connector on DataBricks 13.1 (powered by Spark 3.4.0) will fail because of the below exception. The version of spark connector is 1.1.0.
How to solve it
DBR 13.1 is powered by Spark 3.4.0, but it also includes an additional improvement https://issues.apache.org/jira/browse/SPARK-42968. The improvement introduces a default method
useCommitCoordinatorin interfaceStreamingWrite. BecauseStarRocksWriteimplements both interfaceBatchWriteandStreamingWritewhich both have default methoduseCommitCoordinatoron DBR 13.1, so it will leads default method conflict as the exception says. We could override the method inStarRocksWriteto solve it. see https://www.geeksforgeeks.org/resolving-conflicts-during-multiple-inheritance-in-java/.This improvement is introduced since Spark 3.5, but DBR introduce it in advance, and that's why the connector can run on community Spark 3.4, but failed on DBR 13.1. It seems DBR is not completely compatible with the community Spark.
[1] DataBrick 13.1 release note
Checklist: