pingcap / tispark

TiSpark is built for running Apache Spark on top of TiDB/TiKV
Apache License 2.0
883 stars 244 forks source link

Fix column mismatch #2785

Closed shiyuhang0 closed 2 months ago

shiyuhang0 commented 2 months ago

What problem does this PR solve?

close #2750

fix column mismatch reported by tikv. https://github.com/tikv/tikv/blob/254ceaaa654a70d78919ee5d85f5b4cbe3c444a3/components/tidb_query_executors/src/index_scan_executor.rs#L113

This occurs when the primary_column_ids number is more than columns number in the index scan of DAGRequest. Example:

start_ts_fallback: 451202703803809793
executors {
  tp: TypeIndexScan
  idx_scan {
    table_id: 93
    index_id: 2
    columns {
      column_id: 1
      tp: 15
      collation: -46
      columnLen: 64
      decimal: 0
      flag: 4107
      default_val: "\000"
      pk_handle: false
    }
    columns {
      column_id: 2
      tp: 8
      collation: -63
      columnLen: 20
      decimal: 0
      flag: 4099
      default_val: "\000"
      pk_handle: false
    }
    primary_column_ids: 3
    primary_column_ids: 1
    primary_column_ids: 2
  }
}
executors {
  tp: TypeLimit
  limit {
    limit: 21
  }
}
time_zone_offset: -25200
flags: 0
output_offsets: 1
output_offsets: 0
encode_type: TypeChunk

What is changed and how it works?

Add primary columns to columns in index scan of DAGRequest when pk is command handle.

Add this because TiDB also do like this. Ref: https://github.com/pingcap/tidb/blob/ddcaadbb856f0890e91e4c77991f0d2aa5aa93d0/pkg/planner/core/planbuilder.go#L1515

Check List

Tests

Code changes

Side effects

Related changes

shiyuhang0 commented 2 months ago

/run-all-tests tidb=release-6.1 tikv=release-6.1 pd=release-6.1

shiyuhang0 commented 2 months ago

/run-all-tests tidb=release-6.1 tikv=release-6.1 pd=release-6.1

shiyuhang0 commented 2 months ago

/run-all-tests tidb=release-6.1 tikv=release-6.1 pd=release-6.1

shiyuhang0 commented 2 months ago

/run-all-tests tidb=release-6.1 tikv=release-6.1 pd=release-6.1

shiyuhang0 commented 2 months ago

/run-all-tests tidb=release-6.1 tikv=release-6.1 pd=release-6.1

shiyuhang0 commented 2 months ago

/run-all-tests tidb=release-6.1 tikv=release-6.1 pd=release-6.1

shiyuhang0 commented 2 months ago

/run-all-tests tidb=release-6.1 tikv=release-6.1 pd=release-6.1

shiyuhang0 commented 2 months ago

/run-all-tests tidb=release-6.1 tikv=release-6.1 pd=release-6.1

shiyuhang0 commented 2 months ago

/run-all-tests tidb=release-6.1 tikv=release-6.1 pd=release-6.1

shiyuhang0 commented 2 months ago

/run-all-tests tidb=release-6.1 tikv=release-6.1 pd=release-6.1

ti-chi-bot[bot] commented 2 months ago

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: xuanyu66, zhangyangyu

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files: - ~~[OWNERS](https://github.com/pingcap/tispark/blob/master/OWNERS)~~ [xuanyu66] Approvers can indicate their approval by writing `/approve` in a comment Approvers can cancel approval by writing `/approve cancel` in a comment
ti-chi-bot[bot] commented 2 months ago

[LGTM Timeline notifier]

Timeline:

ti-chi-bot commented 2 months ago

In response to a cherrypick label: new pull request created to branch release-3.1: #2786.

ti-chi-bot commented 2 months ago

In response to a cherrypick label: new pull request created to branch release-3.2: #2787.