pingcap / tispark

TiSpark is built for running Apache Spark on top of TiDB/TiKV
Apache License 2.0
880 stars 244 forks source link

[BUG] The Tispark task failed when injecting one pd, one voter, and one learner failure are injected at the same time(Simulated one AZ failure #2705

Closed Lily2025 closed 1 year ago

Lily2025 commented 1 year ago

Describe the bug

What did you do df=spark.sql("select * from sysbench_1_100000000000.sbtest1") df.write.format('csv').save('ks3://qe-testing/kernel-testing/rel-read-learner-leader-05191158',mode="overwrite")

What do you expect The Tispark task can be succeed when injecting one pd, one voter, and one learner failure are injected at the same time

What happens instead The Tispark task failed when injecting one pd, one voter, and one learner failure are injected at the same time(Simulated one AZ failure image

Spark and TiSpark version info TiDB: v6.5.0 TiSpark: v3.2.1, Spark: 3.3.2

Additional context

Lily2025 commented 1 year ago

/type bug /severity critical /assign shiyuhang0

shiyuhang0 commented 1 year ago

TiSpark config:

park.sql.extensions  org.apache.spark.sql.TiExtensions
spark.tispark.pd.addresses  10.99.182.141:2379
spark.sql.catalog.tidb_catalog  org.apache.spark.sql.catalyst.catalog.TiCatalog
spark.sql.catalog.tidb_catalog.pd.addresses  10.99.182.141:2379
spark.sql.auth.enable false
spark.tispark.replica_read learner,leader