Open SteNicholas opened 9 months ago
Organization: New Baatur
Region: Shen zhen, China
Contact information: 1623735386@qq.com
Usage scenario: Use Apache Celeborn to improve the efficiency and resilience of the Spark/MapReduce engine, providing elastic and efficient management services for intermediate data such as Shuffle data, overflow data, and result data.
Location: Currently, 10+locations have been deployed, with significant stability improvement and performance improvement of over 30%.
Region: Xi'an, China Contact information: 1623735386@qq.com Usage scenario: Use Apache Celeborn to improve the efficiency and resilience of the Spark/MapReduce engine, providing elastic and efficient management services for intermediate data such as Shuffle data, overflow data, and result data. Location: Currently, 10+locations have been deployed, with significant stability improvement and performance improvement of over 30%.
@gaochao0509 Thanks for sharing. Mind also share the company/organization? :-)
Organization: inmobi Group Region: Beijing, China Contact: alex.song@inmobi.com Usage Scenario: We use Apache Celeborn to improve the stability and efficiency of Spark shuffle.
Organization: zto.com Group Region: Shanghai, China Contact: qr7972@gmail.com Usage Scenario: We use Apache Celeborn to improve the stability and efficiency of Spark shuffle
Organization: shuyun Inc
Region: hangzhou, China
Contact: lisoda@yeah.net
Usage Scenario: By using apache celeborn, We realised the separation of storage and compute in HIVE-MR3/SPARK-KYUUBI, which effectively reduces the cost of the server room.
Organization: Zhihu Inc
Region: Beijing, China
Contact: lifulong@zhihu.com
Usage Scenario: We use Apache Celeborn to improve the stability and efficiency of Spark engine shuffle.
Organization: Xiaohongshu Inc
Region: Shanghai, China
Contact: paen@xiaohongshu.com
Usage Scenario: We use Apache Celeborn to improve the stability and efficiency of Spark and Flink(Batch) engine shuffle.
Organization: SHOPEE
Region: Shenzhen, China
Contact: anger.zhu@gmail.com
Usage Scenario: We use Apache Celeborn to improve the stability and efficiency of Spark shuffle.
Organization: DMALL Region: Beijing, China Contact: gang.huang@dmall.com, ming.li@dmall.com, ming.li2@dmall.com Usage Scenario: We use Apache Celeborn to improve the stability and efficiency of Spark shuffle.
Organization: Douban Region: Beijing, China Contact: caofengyu@douban.com Usage Scenario: we use Apache Celeborn to support better Spark dynamic resource allocation in Kubernetes.
Organization: WanFang Data
Region: Beijing, China
Contact: xuxinhao@wanfangdata.com.cn
Usage Scenario: We use Apache Celeborn to improve the stability and efficiency of Spark shuffle on k8s
Organization: Tongcheng Travel(同程旅行).
Region: Jiangsu, China
Contact: www.ly.com
Usage Scenario: we use Apache Celeborn to improve the stability and efficiency of spark shuffle.
Organization: e.g. Pinterest Region: US Contact: @CodingCat Usage Scenario: we use Apache Celeborn to improve the efficiency and elasticity of Spark@K8S
Organization: LinkedIn
Region: US
Contact: @akpatnam25
Usage Scenario: we use Apache Celeborn to improve the stability and efficiency of spark shuffle.
Organization: BOSS Zhipin
Region: Beijing, China
Contact: @liugs0213
Usage Scenario: we use Apache Celeborn to improve the efficiency and elasticity of Spark@K8S
Organization: Stripe
Region: US
Contact: @lohitvijayarenu
Usage Scenario: Apache Celeborn helps Spark workloads with efficient and elastic shuffle. Celeborn will also help in Storage and compute disaggregation for efficient use of resources.
Organization: Xiaomi
Region: Beijing, China
Contact: wangshengjie3@xiaomi.com
Usage Scenario: We use Celeborn to improve the stability and efficiency of shuffle.
Organization: YY
Region: GuangZhou, China
Contact: https://www.yy.com/web/jinhong/#/connect
Usage Scenario: We use Celeborn to improve the stability and efficiency of shuffle.
Organization: WuXianShengHuo Region: HangZhou, China Contact: https://www.weidian.com Usage Scenario: We use Celeborn to improve the stability and efficiency of shuffle.
Search before asking
Motivation
Who's using Apache Celeborn?
The Apache Celeborn community grows with the combined efforts of every user and developer and appreciates every user and developer who uses and cares about Celeborn. We are committed to listening to each one of you to make the community more prosperous.
Therefore, the purpose of this topic is to encourage you, as a user or developer, to leave your real-world scenarios so that we can subsequently explore new and better solutions, schedule the releases, and solve real-world problems based on these scenarios.
We look forward to your comments in the comments space, including but not limited to the following areas.
Regards,
Apache Celeborn Community
Solution
No response
Anything else?
No response
Are you willing to submit a PR?