search

apache / incubator-gluten

Gluten is a middle layer responsible for offloading JVM-based SQL engines' execution to native engines.
https://gluten.apache.org/
Apache License 2.0
1.22k stars 439 forks source link

free(): invalid pointer issue in parquet write with static build #5820

Open JkSelf opened 6 months ago

JkSelf commented 6 months ago

Backend

VL (Velox)

Bug description

When executing a Spark Parquet write unit test on CentOS 7 using a static build approach, the following exception is encountered. The steps to reproduce the issue are as follows：

source ./dev/vcpkg/env.sh
./dev/buildbundle-veloxbe.sh --build_tests=ON --build_benchmarks=ON --enable_s3=ON  --enable_hdfs=ON
mvn -ntp clean install -Pspark-3.2 -Pspark-ut -Pbackends-velox -Pceleborn -Piceberg -Pdelta -DwildcardSuites=org.apache.spark.sql.execution.VeloxParquetWriteSuite

Spark version

Spark-3.2.x

Spark configurations

No.

System information

Velox System Info v0.0.2 Commit: 76c00c34f47c47d008c4c9c90ce31deaf4e01065 CMake Version: 3.27.1 System: Linux-3.10.0-1160.2.2.el7.x86_64 Arch: x86_64 C++ Compiler: /opt/rh/devtoolset-9/root/usr/bin/c++ C++ Compiler Version: 9.3.1 C Compiler: /opt/rh/devtoolset-9/root/usr/bin/cc C Compiler Version: 9.3.1 CMake Prefix Path: /usr/local;/usr;/;/home/jk/incubator-gluten/dev/vcpkg/.vcpkg/downloads/tools/cmake-3.27.1-linux/cmake-3.27.1-linux-x86_64;/usr/local;/usr/X11R6;/usr/pkg;/opt

Relevant logs

*** Error in `/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.402.b06-1.el7_9.x86_64/jre/bin/java': free(): invalid pointer: 0x00007f61063ed1a0 ***
/lib64/libc.so.6(+0x81329)[0x7f61af5bf329]
/tmp/gluten-b3a2e1f9-4627-4c08-8963-e97d46453e09/jni/d379ebe2-d7e2-4ef0-961a-3b21e11ffef3/gluten-8772818901135708910/libvelox.so(+0x4eb231c)[0x7f5ecd4cf31c]
/tmp/gluten-b3a2e1f9-4627-4c08-8963-e97d46453e09/jni/d379ebe2-d7e2-4ef0-961a-3b21e11ffef3/gluten-8772818901135708910/libvelox.so(_ZN8facebook5velox7parquet5arrow5arrow14SchemaManifest4MakeEPKNS2_16SchemaDescriptorERKSt10shared_ptrIKN5arrow16KeyValueMetadataEERKNS2_21ArrowReaderPropertiesEPS4_+0x4c3)[0x7f5ecd4d09f3]
/tmp/gluten-b3a2e1f9-4627-4c08-8963-e97d46453e09/jni/d379ebe2-d7e2-4ef0-961a-3b21e11ffef3/gluten-8772818901135708910/libvelox.so(_ZN8facebook5velox7parquet5arrow5arrow10FileWriter4MakeEPN5arrow10MemoryPoolESt10unique_ptrINS2_17ParquetFileWriterESt14default_deleteIS9_EESt10shared_ptrINS5_6SchemaEESD_INS2_21ArrowWriterPropertiesEEPS8_IS4_SA_IS4_EE+0x236)[0x7f5ecd4bec96]
/tmp/gluten-b3a2e1f9-4627-4c08-8963-e97d46453e09/jni/d379ebe2-d7e2-4ef0-961a-3b21e11ffef3/gluten-8772818901135708910/libvelox.so(_ZN8facebook5velox7parquet5arrow5arrow10FileWriter4OpenERKN5arrow6SchemaEPNS5_10MemoryPoolESt10shared_ptrINS5_2io12OutputStreamEESB_INS2_16WriterPropertiesEESB_INS2_21ArrowWriterPropertiesEE+0x2d7)[0x7f5ecd4bf4e7]
/tmp/gluten-b3a2e1f9-4627-4c08-8963-e97d46453e09/jni/d379ebe2-d7e2-4ef0-961a-3b21e11ffef3/gluten-8772818901135708910/libvelox.so(_ZN8facebook5velox7parquet6Writer5flushEv+0x73d)[0x7f5ecd4b414d]
/tmp/gluten-b3a2e1f9-4627-4c08-8963-e97d46453e09/jni/d379ebe2-d7e2-4ef0-961a-3b21e11ffef3/gluten-8772818901135708910/libvelox.so(_ZN8facebook5velox7parquet6Writer5closeEv+0x1c)[0x7f5ecd4b37ec]
/tmp/gluten-b3a2e1f9-4627-4c08-8963-e97d46453e09/jni/d379ebe2-d7e2-4ef0-961a-3b21e11ffef3/gluten-8772818901135708910/libgluten.so(Java_org_apache_gluten_datasource_DatasourceJniWrapper_close+0x19d)[0x7f6104e56e8d]
JkSelf commented 6 months ago

https://github.com/facebookincubator/velox/discussions/9875