-
### Backend
VL (Velox)
### Bug description
Report a bug per Weiting's recomendation.
Steps to produce
1. Set up spark to run with gluten.
2. Run this script with `./run-etl-tpch.sh 0 1` -> ht…
-
### Backend
VL (Velox)
### Bug description
Test TPCDS query-24.SQL 3T, Spark driver memory overflow, analyze through heapdump file analyze the reason,maybe the cache plan ，4000+ GlutenPartiton have…
-
### Backend
VL (Velox)
### Bug description
one common issue of Gluten is that the task is killed by yarn. Currently Gluten has some memory allocation like std::vector isn't tracked by spark's memor…
-
First of all, great work!
Would be really neat if a custom allocator could be used instead of `new []` - as this library is most likely used in context of some web services and decent memory parame…
-
Very nice initiative ! I'll have a look at how to integrate into gRPC and use it in [Pulsar-gRPC](https://github.com/cbornet/pulsar-grpc)
Since `parseFrom` does zero-copy, I understand that the `B…
-
I changed the base image from gcr.io spark:3.1.1 to spark:3.5.0 and built my dockerfile from there where it has my DataAnalyticsReporting.jar
This is the error when the spark-operator tries to star…
-
### Description
Velox backend provides 2-level file cache (`AsyncDataCache` and `SsdCache`) and we have enabled it in [PR](https://github.com/apache/incubator-gluten/pull/1076/files), using a dedic…
yma11 updated
5 months ago
-
Off-heap allocation has to be understand as direct memory allocation.
It can be measured with the help of a MBean. Undersatand the implemementations of @MeasureHeapAllocation, @ExpectedHeapAllocatio…
-
### Description
### Backend
Velox
### Perf description
When switching from tpch 1T to tpch 3T, the Shuffle Read time of q21 will significantly increase, and the system's disk/network resources are…
-
I wouldn't say this is crucial, but it seems if block sizes reach `512 kB` or `1 MB` you might want to move them from Task statedata to being in Mnesia (or maybe just an ETS table because only a singl…