dvonthenen
commented
8 years ago Deep Dive with Mesos and Persistent Storage for Applications
- Introduction
- Presenter
- Agenda
- Quick Review of Mesos
- Describe Mesos
- Singly managed resource pool
- Scheduler/Dispatcher of Applications
- Mesos
- Master Nodes (Zookeeper Quorum)
- Agent (Slave) Nodes
- Application Support
- Generic
- Framework
- Application Examples
- Frameworks and Two Layer Scheduling
- What is a Framework?
- Schedulers
- Executors
- Framework Architecture
- What are the moving parts?
- Who is talking to who? (Interaction Diagram)
- What does a Framework buy me?
- Two Layer Scheduling
- Use cases from other popular frameworks
- GPU
- Disperse application deployment
- Etc
- Framework Examples
- Production Applications
- Using Containers for Apps
- Containers Stateless
- Applications need state somewhere!
- Example applications needing state
- Applications without state?
- Hmmmm...
- How to get state?
- Mesos... local disk
- Docker... local disk
- Better way?
- Persistent External Storage
- mesos-module-dvdi
- REX-Ray
- Demo
- Configuration Description of Demo (AWS)
- Demo 1: Production Website
- Start Deploy and Describe Setup (MVC)
- UI (Golang)
- Business Logic - REST API (Golang)
- Postgres (Model)
- Disaster!
- Node Failure
- Disk Failure
- Downtime (or performance hit) to RAID rebuild
- Network failure or partitioning event!
- Answer: External Storage!
- Demo 2: Elastic Search Framework using External Storage
- Start Deploy and Describe Setup
- Elastic Search Mesos Framework
- 3 Node Elastic Search Cluster
- Disaster!
- Answer: External Storage!
- Node recovers from where it left off (quicker recovery)
- No node rebuild (hardware rebuild fast, software slow)
- Conclusions
clintkitson
Persistent applications that can be complex to operate and scale tend to be perfect for Apache Mesos. Internal direct attached storage and external storage are both options to run your applications. This talk will outline patterns for using these to allow deployment of managed frameworks and tasks, while maintaining fault tolerance and scalability.