search

apache / incubator-gluten

Gluten is a middle layer responsible for offloading JVM-based SQL engines' execution to native engines.
https://gluten.apache.org/
Apache License 2.0
1.22k stars 437 forks source link

[VL][1.2] Type mismatch error in VectorHasher #7652

Open wForget opened 4 weeks ago

wForget commented 4 weeks ago

Backend

VL (Velox)

Bug description

Type mismatch error in VectorHasher.

Spark version

Spark-3.5.x

Spark configurations

No response

System information

No response

Relevant logs

org.apache.gluten.exception.GlutenException: Exception: VeloxRuntimeError
Error Source: RUNTIME
Error Code: INVALID_STATE
Reason: Type mismatch: VARCHAR vs. VARBINARY
Retriable: False
Expression: type_->kindEquals(vector.type())
Context: Operator: PartialAggregation[1] 1
Function: decode
File: /data/workspace/gluten-deploy-dist/ep/build-velox/build/velox_ep/./velox/exec/VectorHasher.h
Line: 168
Stack trace:
# 0  _ZN8facebook5velox7process10StackTraceC1Ei
# 1  _ZN8facebook5velox14VeloxExceptionC1EPKcmS3_St17basic_string_viewIcSt11char_traitsIcEES7_S7_S7_bNS1_4TypeES7_
# 2  _ZN8facebook5velox6detail14veloxCheckFailINS0_17VeloxRuntimeErrorERKSsEEvRKNS1_18VeloxCheckFailArgsET0_
# 3  _ZN8facebook5velox4exec13BaseHashTable20prepareForGroupProbeERNS1_10HashLookupERKSt10shared_ptrINS0_9RowVectorEERNS0_17SelectivityVectorEba
# 4  _ZN8facebook5velox4exec11GroupingSet21addInputForActiveRowsERKSt10shared_ptrINS0_9RowVectorEEb
# 5  _ZN8facebook5velox4exec11GroupingSet8addInputERKSt10shared_ptrINS0_9RowVectorEEb
# 6  _ZN8facebook5velox4exec15HashAggregation8addInputESt10shared_ptrINS0_9RowVectorEE
# 7  _ZN8facebook5velox4exec6Driver11runInternalERSt10shared_ptrIS2_ERS3_INS1_13BlockingStateEERS3_INS0_9RowVectorEE
# 8  _ZN8facebook5velox4exec6Driver4nextERSt10shared_ptrINS1_13BlockingStateEE
# 9  _ZN8facebook5velox4exec4Task4nextEPN5folly10SemiFutureINS3_4UnitEEE
# 10 _ZN6gluten24WholeStageResultIterator4nextEv
# 11 Java_org_apache_gluten_vectorized_ColumnarBatchOutIterator_nativeHasNext
# 12 0x00007ff82ebb76c7

    at org.apache.gluten.vectorized.ColumnarBatchOutIterator.nativeHasNext(Native Method)
    at org.apache.gluten.vectorized.ColumnarBatchOutIterator.hasNextInternal(ColumnarBatchOutIterator.java:61)
    at org.apache.gluten.vectorized.GeneralOutIterator.hasNext(GeneralOutIterator.java:37)
    ... 33 more

    at org.apache.gluten.vectorized.ColumnarBatchOutIterator.nativeHasNext(Native Method)
    at org.apache.gluten.vectorized.ColumnarBatchOutIterator.hasNextInternal(ColumnarBatchOutIterator.java:61)
    at org.apache.gluten.vectorized.GeneralOutIterator.hasNext(GeneralOutIterator.java:37)
    ... 21 more
wForget commented 4 weeks ago

Reproduce:

create table t1 (a string) stored as parquet;

-- insert a binary value to t1 location
INSERT OVERWRITE DIRECTORY 'hdfs://XXXXX/t1' USING parquet select cast('test' as binary) as a;

-- got type mismatch error
select count(1) from t1 group by a;
LoseYSelf commented 4 weeks ago

hello,i met task error,but not print Stack trace as you like, had any config i miss? it print like 

Retriable: False
Function: runInternal
File: /data/app/gluten/ep/build-velox/build/velox_ep/velox/exec/Driver.cpp
Line: 742
Stack trace:
# 0  
# 1  
# 2  
# 3  
# 4  
# 5  
# 6  
# 7  
# 8  

    at org.apache.gluten.vectorized.ColumnarBatchOutIterator.nativeHasNext(Native Method)
    at org.apache.gluten.vectorized.ColumnarBatchOutIterator.hasNextInternal(ColumnarBatchOutIterator.java:61)
    at org.apache.gluten.vectorized.GeneralOutIterator.hasNext(GeneralOutIterator.java:37)
    ... 46 more

thx

wForget commented 4 weeks ago

@LoseYSelf Are you using gluten 1.2.0? Maybe you need port #7038

LoseYSelf commented 4 weeks ago

@LoseYSelf Are you using gluten 1.2.0? Maybe you need port #7038

thx