State of the container eco-system with persistent applications

[clintkitson]

First generation runtimes for containers assumed the workload inside the container would be stateless and ephemeral. But, most useful systems require storage of state somewhere. With the progression of container platforms from Mesos and Docker, you can easily run your stateful applications such as databases inside of containers. This session will cover the current state of persistent storage, containers and schedulers, including future directions in this arena.

[clintkitson]

Associated blog post https://github.com/emccode/EMCW2016/issues/24

[cantbewong]

Presentation outline

Where did containers start - generation 1

• SIngle host scope
  • Linux Containers
  • Docker

What were some limitations with the first generation of containers

• Not scalable - single node at a time
• high management cost
• difficult to maintain high utilization
• single point of failure
• like "ESX without a vCenter"

The present - containers generation 2 enhancements

• Schedulers
• external storage support

Schedulers

• Let you manage a pool of cluster hosts, rather than one at a time

Schedulers vary in

• Upper limit to cluster size
• Technique of cluster management
• Degree and Technique of Container/Application management
• External storage enablement, and management

External storage support - Why it's important

• Enables
  • Transaction databases (SQL)
  • NoSql databases
  • distributed filesystems

Scheduler Comparison table

• Mesos
• Kubernetes
• Cloud Foundry
• Swarm

The future

• Schedule management of external storage
  • not just tolerance, but monitoring of resource capacity and utilization
• Central orchestration across multiple schedulers
• Offer pools, with volume suballocation
• "mulltitenancy" by schedulers
  • resource allocations
  • authorization based reporting

Polly project overview

[clintkitson]

• gen1
• what is the container eco-system
• state of eco-system with non-persistent applications - schedulers and runtimes galore
• who cares about persistence with containers?
• composable architecture
• codify how different schedulers think about consuming storage - implicit/explicit?
• different consumption models, different schedulers, different container runtimes
• our view of the world
  • shift burden of support to vendors
  • want native support of volume functionality for runtimes and schedulers
  • cluster-level context
  • explicit consumption of storage
• demo of serving volumes across mesos,docker,kubernetes

[clintkitson]

Demo idea 1 https://github.com/emccode/EMCW2016/issues/27

[clintkitson]

Demo idea 2 https://github.com/emccode/EMCW2016/issues/28

[clintkitson]

Demo idea https://github.com/emccode/EMCW2016/issues/33

[cantbewong]

copy draft ppt to google drive.

Note this has way too many slides for allotted time, but I deemed it better to have too many and let reviewers comment on what can be pruned.

Trouble is this is a complex subject to cram into a < 60 min presentation

[cantbewong]

posted completed deck to drive