search

pingcap / tispark

TiSpark is built for running Apache Spark on top of TiDB/TiKV
Apache License 2.0
880 stars 243 forks source link

[BUG] TiSpark scan TiKV partition table crash #2773

Closed qumingxing closed 1 month ago

qumingxing commented 4 months ago

TiDB version：v7.5.1 TiSpark version : 3.2_2.12-3.2.2

7.5.1 TiFlash enable Storage and Compute Architecture and TiSpark scan TiKV partition table failed. none-partition tables are normal

Exception: Caused by: com.pingcap.tikv.exception.RegionTaskException: Handle region task failed: at com.pingcap.tikv.operation.iterator.DAGIterator.process(DAGIterator.java:233) at com.pingcap.tikv.operation.iterator.DAGIterator.lambda$submitTasks$1(DAGIterator.java:91) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) ... 3 more Caused by: com.pingcap.tikv.exception.SelectException: [FLASH:Table:NotExists] Table doesn't exist, keyspace=4294967295 table_id=411246 at com.pingcap.tikv.region.RegionStoreClient.doCoprocessor(RegionStoreClient.java:766)

sunxiaoguang commented 3 months ago

Unfortunately, Current TiSpark implementation doesn't support TiDB version greater than 6.x. https://docs.pingcap.com/tidb/stable/tispark-overview#choose-tispark-version

There will be a new architecture coming out to make TiSpark as thin as a simple adaptor on top of JDBC connector to make it support newer version of TiDB without extra development.

github-actions[bot] commented 2 months ago

This issue is stale because it has been open for 30 days with no activity.

github-actions[bot] commented 1 month ago

This issue was closed because it has been inactive for 14 days since being marked as stale.