search

apache / celeborn

Apache Celeborn is an elastic and high-performance service for shuffle and spilled data.
https://celeborn.apache.org/
Apache License 2.0
860 stars 351 forks source link

Who is using Apache Celeborn? #2140

Open SteNicholas opened 9 months ago

SteNicholas commented 9 months ago

Search before asking

Motivation

Who's using Apache Celeborn?

The Apache Celeborn community grows with the combined efforts of every user and developer and appreciates every user and developer who uses and cares about Celeborn. We are committed to listening to each one of you to make the community more prosperous.

Therefore, the purpose of this topic is to encourage you, as a user or developer, to leave your real-world scenarios so that we can subsequently explore new and better solutions, schedule the releases, and solve real-world problems based on these scenarios.

We look forward to your comments in the comments space, including but not limited to the following areas.

Regards,

Apache Celeborn Community

Solution

No response

Anything else?

No response

Are you willing to submit a PR?

gaochao0509 commented 9 months ago

Organization: New Baatur Region: Shen zhen, China Contact information: 1623735386@qq.com Usage scenario: Use Apache Celeborn to improve the efficiency and resilience of the Spark/MapReduce engine, providing elastic and efficient management services for intermediate data such as Shuffle data, overflow data, and result data. Location: Currently, 10+locations have been deployed, with significant stability improvement and performance improvement of over 30%. Logo

carp84 commented 9 months ago

Region: Xi'an, China Contact information: 1623735386@qq.com Usage scenario: Use Apache Celeborn to improve the efficiency and resilience of the Spark/MapReduce engine, providing elastic and efficient management services for intermediate data such as Shuffle data, overflow data, and result data. Location: Currently, 10+locations have been deployed, with significant stability improvement and performance improvement of over 30%.

@gaochao0509 Thanks for sharing. Mind also share the company/organization? :-)

cxzl25 commented 9 months ago

trip_group_logo

big-loser commented 9 months ago

Organization: inmobi Group Region: Beijing, China Contact: alex.song@inmobi.com Usage Scenario: We use Apache Celeborn to improve the stability and efficiency of Spark shuffle.

RuiQin7 commented 9 months ago

Organization: zto.com Group Region: Shanghai, China Contact: qr7972@gmail.com Usage Scenario: We use Apache Celeborn to improve the stability and efficiency of Spark shuffle

1

BsoBird commented 9 months ago

Organization: shuyun Inc Region: hangzhou, China Contact: lisoda@yeah.net Usage Scenario: By using apache celeborn, We realised the separation of storage and compute in HIVE-MR3/SPARK-KYUUBI, which effectively reduces the cost of the server room. image

SteNicholas commented 9 months ago
jiaoqingbo commented 9 months ago
lifulong commented 9 months ago

Organization: Zhihu Inc Region: Beijing, China Contact: lifulong@zhihu.com Usage Scenario: We use Apache Celeborn to improve the stability and efficiency of Spark engine shuffle. image

TongWei1105 commented 9 months ago

Organization: Xiaohongshu Inc Region: Shanghai, China Contact: paen@xiaohongshu.com Usage Scenario: We use Apache Celeborn to improve the stability and efficiency of Spark and Flink(Batch) engine shuffle. xiaohongshu

onebox-li commented 9 months ago
AngersZhuuuu commented 9 months ago

Organization: SHOPEE Region: Shenzhen, China Contact: anger.zhu@gmail.com Usage Scenario: We use Apache Celeborn to improve the stability and efficiency of Spark shuffle. image

lianneli commented 8 months ago

Organization: DMALL Region: Beijing, China Contact: gang.huang@dmall.com, ming.li@dmall.com, ming.li2@dmall.com Usage Scenario: We use Apache Celeborn to improve the stability and efficiency of Spark shuffle.

B端业务-横版-蓝色
camper42 commented 7 months ago

Organization: Douban Region: Beijing, China Contact: caofengyu@douban.com Usage Scenario: we use Apache Celeborn to support better Spark dynamic resource allocation in Kubernetes.

豆瓣logo

wfxxh commented 6 months ago

Organization: WanFang Data Region: Beijing, China Contact: xuxinhao@wanfangdata.com.cn Usage Scenario: We use Apache Celeborn to improve the stability and efficiency of Spark shuffle on k8s image

wxplovecc commented 6 months ago

Organization: Tongcheng Travel（同程旅行）. Region: Jiangsu, China Contact: www.ly.com Usage Scenario: we use Apache Celeborn to improve the stability and efficiency of spark shuffle. image

CodingCat commented 6 months ago

Organization: e.g. Pinterest Region: US Contact: @CodingCat Usage Scenario: we use Apache Celeborn to improve the efficiency and elasticity of Spark@K8S

image

akpatnam25 commented 6 months ago

Organization: LinkedIn Region: US Contact: @akpatnam25 Usage Scenario: we use Apache Celeborn to improve the stability and efficiency of spark shuffle. image

liugs0213 commented 5 months ago

Organization: BOSS Zhipin Region: Beijing, China Contact: @liugs0213 Usage Scenario: we use Apache Celeborn to improve the efficiency and elasticity of Spark@K8S image

lohitvijayarenu commented 1 month ago

Organization: Stripe Region: US Contact: @lohitvijayarenu Usage Scenario: Apache Celeborn helps Spark workloads with efficient and elastic shuffle. Celeborn will also help in Storage and compute disaggregation for efficient use of resources. image

wangshengjie123 commented 1 month ago

Organization: Xiaomi Region: Beijing, China Contact: wangshengjie3@xiaomi.com Usage Scenario: We use Celeborn to improve the stability and efficiency of shuffle. 小米办公20240809-184309

ASiegeLion commented 3 weeks ago

Organization: YY Region: GuangZhou, China Contact: https://www.yy.com/web/jinhong/#/connect Usage Scenario: We use Celeborn to improve the stability and efficiency of shuffle. image

jinyuchen724 commented 3 weeks ago

Organization: WuXianShengHuo Region: HangZhou, China Contact: https://www.weidian.com Usage Scenario: We use Celeborn to improve the stability and efficiency of shuffle.

image