Hadoop, MPI 등 여러 다양한 클러스터 컴퓨팅 프레임워크 간에 리소스를 공유할 수 있는 플랫폼에 관한 연구
Abstract (요약) 🕵🏻♂️
We present Mesos, a platform for sharing commodity clusters between multiple diverse cluster computing frameworks, such as Hadoop and MPI. Sharing improves cluster utilization and avoids per-framework data replication. Mesos shares resources in a fine-grained manner, allowing frameworks to achieve data locality by taking turns reading data stored on each machine. To support the sophisticated schedulers of today’s frameworks, Mesos introduces a distributed two-level scheduling mechanism called resource offers. Mesos decides how many resources to offer each framework, while frameworks decide which resources to accept and which computations to run on them. Our results show that Mesos can achieve near-optimal data locality when sharing the cluster among diverse frameworks, can scale to 50,000 (emulated) nodes, and is resilient to failures.
sypark9646
commented
1 year ago
어떤 내용의 논문인가요? 👋
Hadoop, MPI 등 여러 다양한 클러스터 컴퓨팅 프레임워크 간에 리소스를 공유할 수 있는 플랫폼에 관한 연구
Abstract (요약) 🕵🏻♂️
We present Mesos, a platform for sharing commodity clusters between multiple diverse cluster computing frameworks, such as Hadoop and MPI. Sharing improves cluster utilization and avoids per-framework data replication. Mesos shares resources in a fine-grained manner, allowing frameworks to achieve data locality by taking turns reading data stored on each machine. To support the sophisticated schedulers of today’s frameworks, Mesos introduces a distributed two-level scheduling mechanism called resource offers. Mesos decides how many resources to offer each framework, while frameworks decide which resources to accept and which computations to run on them. Our results show that Mesos can achieve near-optimal data locality when sharing the cluster among diverse frameworks, can scale to 50,000 (emulated) nodes, and is resilient to failures.
이 논문을 읽어서 무엇을 배울 수 있는지 알려주세요! 🤔
이 논문을 제대로 읽었을 때 어떤 지식을 얻을 수 있을까요?
같이 읽어보면 좋을 만한 글이나 이슈가 있을까요?
만약에 있다면 자유롭게 작성해 주세요!
레퍼런스의 URL을 알려주세요! 🔗
markdown 으로 축약하지 말고, 원본 링크 그대로 그냥 적어주세요!