-
@hiboyang Have you tried using remote shuffle service with spark operator? (spark on K8s operator)?
I tested it with the client jar in my 'SparkApplication' image and it works as expected.
Alth…
-
Loading large dataset gives an error: "MemoryError: Unable to allocate 202. GiB for an array with shape (27084389376,) and data type int64"
```
self.num_shards
237460
self.num_samples
2708438…
-
#### What would you like to be added:
Add the Apache Uniffle (incubating) as a part of Ecosystem.
Our repo is https://github.com/apache/incubator-uniffle
More details https://uniffle.apache.org/bl…
jerqi updated
6 months ago
-
1. Does it support both remote shuffle service and dynamic resource allocation?
2. Does the enabling of `spark.shuffle.service.enabled` have an impact on the shuffle service?
-
i'm noticing running some spark apps that produce 11TB of shuffle data on external shuffle service, that they produce closer to 18TB of shuffle data on remote shuffle service. is some write amplificat…
cpd85 updated
2 years ago
-
Hey,
First of all thank you so much for this awesome add-on works flawlessly totally in love with this so recently tried mqtt aswell but was wondering how do I get the album cover? i have album cove…
-
### Code of Conduct
- [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)
### Search before asking
- [X] I have searched in the [issues](http…
-
### Expected Behavior
The order of elements should be preserved when calling `org.gradle.caching.BuildCacheServiceFactory.Describer#config(String, String)` multiple times.
### Current Behavior
…
vlsi updated
9 months ago
-
I see there are 2 branches K8 and rss-k8, which branch should be used for building jar and image for remote shuffle service for K8 environment?
-
### Code of Conduct
- [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)
### Search before asking
- [X] I have searched in the [issue…