The GlutenConfig is not initialized by sparkConf when calling setPredefinedConfigs. As a result, GlutenConfig.getConf.enableGluten and GlutenConfig.getConf.enableGlutenCostEvaluator in setPredefinedConfigs only retrieve their default values. To ensure correct configuration, we should retrieve these settings directly from sparkConf.
Spark version
None
Spark configurations
No response
System information
Gluten Version: 1.3.0-SNAPSHOT
Commit: e204dc5f06eef147d88b5ae6e8532b3902b19153
CMake Version: 3.20.2
System: Linux-5.4.119-19-0009.11
Arch: x86_64
CPU Name: Model name: Intel(R) Xeon(R) Platinum 8372HC CPU @ 3.40GHz
BIOS Model name: 3.0
C++ Compiler: /usr/local/bin/g++
C++ Compiler Version: 10.3.1
C Compiler: /usr/bin/cc
C Compiler Version: 8.5.0
CMake Prefix Path: /usr/local;/usr;/;/usr;/usr/local;/usr/X11R6;/usr/pkg;/opt
Backend
VL (Velox)
Bug description
The GlutenConfig is not initialized by sparkConf when calling setPredefinedConfigs. As a result, GlutenConfig.getConf.enableGluten and GlutenConfig.getConf.enableGlutenCostEvaluator in setPredefinedConfigs only retrieve their default values. To ensure correct configuration, we should retrieve these settings directly from sparkConf.
Spark version
None
Spark configurations
No response
System information
Gluten Version: 1.3.0-SNAPSHOT Commit: e204dc5f06eef147d88b5ae6e8532b3902b19153 CMake Version: 3.20.2 System: Linux-5.4.119-19-0009.11 Arch: x86_64 CPU Name: Model name: Intel(R) Xeon(R) Platinum 8372HC CPU @ 3.40GHz BIOS Model name: 3.0 C++ Compiler: /usr/local/bin/g++ C++ Compiler Version: 10.3.1 C Compiler: /usr/bin/cc C Compiler Version: 8.5.0 CMake Prefix Path: /usr/local;/usr;/;/usr;/usr/local;/usr/X11R6;/usr/pkg;/opt
Relevant logs
No response