-
### Backend
VL (Velox)
### Bug description
I am trying to set `spark.gluten.memory.dynamic.offHeap.sizing.enabled=true"`, but OOM exception occurs.
spark configuration :
```
spark.executor.memor…
-
### Backend
VL (Velox)
### Bug description
offheap is 8.5GB, no fallback. When an operator researves ~6.5G the spill is triggered. ~0.75% of offheap size.
@zhztheplayer
-
It's not an issue, rather a question: I,m trying to make offheap object, that contains offheap array
```
@JillegalAware
public class HashVO extends DefaultVO {
private HashVONode[] voarray;
p…
-
### env：
```
host
os：CentOS Linux release 8.2.2004 (Core)
kernel：4.18.0-193.el8.x86_64
memory：1T
jdk：1.8
maven：3.9.6
spark：3.4
scala：2.12
container：
os：CentOS Linux release 7.4.1708 (Core)
…
-
in the comment of last issue, you said
However there are some problems for offheap part of project due to compact phase of GC.
These problems can be handled by VM arguments but i am working on a bett…
-
**Describe the bug**
I run spark with blaze + celeborn on k8s always OOM
**To Reproduce**
My environment:
spark version: 3.3.3 with celeborn DRA patch
blaze version: blaze-engine-spa…
-
currently metadata store is kept in memory.
alternative is to keep metadata store offheap (mem-map-file?)
option can be to be able to choose/change/plug a store
-
## Implement stage-level resourceProfile auto-adjust framework to avoid oom
### Backgroud
In our production environment, we suffer a lot from gluten jobs throwing heap OOM exception occasionally. …
-
So far `ColumnarBuildSideRelation` is allocated on Spark JVM heap memory.
```
case class ColumnarBuildSideRelation(output: Seq[Attribute], batches: Array[Array[Byte]]) {
...
}
```
It appea…
-
An h2 server running under relatively low load (~2% CPU and 50m database) constantly grows in RAM. In spite of that all limits are specified, the application RSS grows with the speed depending on syst…