apache / incubator-uniffle

Uniffle is a high performance, general purpose Remote Shuffle Service.
https://uniffle.apache.org/
Apache License 2.0
388 stars 149 forks source link

[FEATURE] Add support about setting extra java properties in client side #2190

Closed zuston closed 1 month ago

zuston commented 1 month ago

Code of Conduct

Search before asking

Describe the feature

In rss client side, it's necessary to set the extra java options by the config options, like the spark conf: spark.executor.extraJavaOptions.

For example, I want to set the io.grpc.netty.shaded.io.netty.eventLoopThreads to avoid too much threads, in current codebase, I have to set it by the spark side javaOptions. This is not easy to make all jobs applied of this option.

Samely, if we have the rss.extra.JavaSystemProperties config, this could be applied and controlled by the rss side rather than spark service.

Motivation

No response

Describe the solution

No response

Additional context

No response

Are you willing to submit PR?

zuston commented 1 month ago

cc @maobaolong