search

apache / incubator-gluten

Gluten is a middle layer responsible for offloading JVM-based SQL engines' execution to native engines.
https://gluten.apache.org/
Apache License 2.0
1.13k stars 409 forks source link

[VL] scan time metric is 0 #5606

Open zhouyuan opened 3 months ago

zhouyuan commented 3 months ago

Backend

VL (Velox)

Bug description

The total time scan metric seems to be wrong image

Spark version

None

Spark configurations

No response

System information

No response

Relevant logs

No response

zhouyuan commented 3 months ago

CC @surnaik

ulysses-you commented 3 months ago

raw input bytes related to https://github.com/facebookincubator/velox/pull/8545 ? cc @JkSelf

surnaik commented 3 months ago

Investigating - https://github.com/facebookincubator/velox/blob/9fde0a2c4bc392d71524e43472604a506b02b6b9/velox/dwio/common/InputStream.cpp#L86

stats_ is NULL

surnaik commented 3 months ago

Passing the received ioStats to https://github.com/facebookincubator/velox/blob/9fde0a2c4bc392d71524e43472604a506b02b6b9/velox/dwio/common/DirectBufferedInput.h#L115 fixes the issue.

JkSelf commented 3 months ago

@zhouyuan @surnaik @ulysses-you The issues with incorrect rawInputBytes and scan time metrics are addressed in the pull request at https://github.com/facebookincubator/velox/pull/9801.

image