search

oap-project / gazelle_plugin

Native SQL Engine plugin for Spark SQL with vectorized SIMD optimizations.
Apache License 2.0
256 stars 77 forks source link

OOM when insert hash map in agg #1128

Open jackylee-ch opened 1 year ago

jackylee-ch commented 1 year ago

Describe the bug The status returned by GetOrInsert is Invalid: Error during calling Java code from native code. The OOM stack is blew.

0# arrow::jniutil::CheckException(JNIEnv_*) in /tmp/jnilib-6179546608607659183.tmp
 1# 0x00007F956327A747 in /tmp/jnilib-6179546608607659183.tmp
 2# arrow::jniutil::ReservationListenableMemoryPool::Impl::UpdateReservation(long) in /tmp/jnilib-6179546608607659183.tmp
 3# arrow::jniutil::ReservationListenableMemoryPool::Allocate(long, unsigned char**) in /tmp/jnilib-6179546608607659183.tmp
 4# arrow::AllocateResizableBuffer(long, arrow::MemoryPool*) in /tmp/spark_columnar_plugin_7541932432899871978/libarrow.so
 5# arrow::BufferBuilder::Resize(long, bool) in /tmp/spark_columnar_plugin_7541932432899871978/libspark_columnar_jni.so
 6# 0x00007F956A3CB015 in /tmp/spark_columnar_plugin_7541932432899871978/libspark_columnar_jni.so
 7# sparkcolumnarplugin::precompile::StringHashMap::GetOrInsert(void const*, int, void (*)(int), void (*)(int), int*) in /tmp/spark_columnar_plugin_7541932432899871978/libspark_columnar_jni.so
 8# sparkcolumnarplugin::codegen::arrowcompute::extra::HashAggregateKernel::Impl::HashAggregateResultIterator<arrow::StringType, void>::ProcessAndCacheOne(std::vector<std::shared_ptr<arrow::Array>, std::allocator<std::shared_ptr<arrow::Array> > > const&, std::shared_ptr<arrow::Array> const&) in /tmp/spark_columnar_plugin_7541932432899871978/libspark_columnar_jni.so
 9# Java_com_intel_oap_vectorized_BatchIterator_nativeProcessAndCacheOne in /tmp/spark_columnar_plugin_7541932432899871978/libspark_columnar_jni.so
10# 0x00007F95AEFC3835