search

apache / celeborn

Apache Celeborn is an elastic and high-performance service for shuffle and spilled data.
https://celeborn.apache.org/
Apache License 2.0
862 stars 351 forks source link

[CELEBORN-1566] Update docs about using HDFS #2683

Closed zhaohehuhu closed 2 weeks ago

zhaohehuhu commented 1 month ago

What changes were proposed in this pull request?

update client deployment doc to include the param (spark.celeborn.storage.activeTypes)

Why are the changes needed?

Just provide a hint for users, otherwise they may miss this param.

Does this PR introduce any user-facing change?

No

How was this patch tested?

Yes

zhaohehuhu commented 1 month ago

plz review this PR. Thanks @FMX

FMX commented 4 weeks ago

It looks like the UT failures are not relevant to this PR. The README file should be updated too.

zhaohehuhu commented 4 weeks ago

It looks like the UT failures are not relevant to this PR. The README file should be updated too.

Done. Thanks.

RexXiong commented 3 weeks ago

seems duplicated with https://github.com/apache/celeborn/pull/2675, If you don't mind, we may need to close this PR.

zhaohehuhu commented 3 weeks ago

seems duplicated with #2675, If you don't mind, we may need to close this PR.

OK. It's ok for me.

bowenliang123 commented 3 weeks ago

This is not duplicated with #2675 . Updating different part of the docs , and for different purposes.

RexXiong commented 2 weeks ago

This is not duplicated with #2675 . Updating different part of the docs , and for different purposes.

Thank you for clarifying, you are correct.

RexXiong commented 2 weeks ago

merge to main(v0.6.0)