OSCON 2016 Guest Blog Post

[amandakatona]

Our OSCON sponsorship entitles us to a guest blog post. That means the blog post would be written by someone at EMC {code}, and then edited by OSCON. OSCON needs to know 3/23 if you have any particular ideas about the topic.

Here are some general guidelines:

• Post topics should offer new knowledge to readers; avoid posts that drift toward promoting a product.
• Posts should be around 500 words.
• OSCONT only promotes/links to things in posts/articles that are relevant to the content and useful to the reader.

Here are some notes about workflow:

• Post should be in Google doc format if possible.
• Images and all needed permission need to be ready 2 weeks before posting window.
• Author's head shot (highest resolution possible––1400px wide minimum) and Twitter handles should be turned in with the full draft 2 weeks before the posting window.

Here are a couple examples of guest posts we've done in the past: http://radar.oreilly.com/2015/06/driven-by-the-internet-of-things-a-mountain-bike-with-a-voice.html http://radar.oreilly.com/2014/06/the-internet-of-things-inflection-point.html

Full draft due to OSCON by 4/5.

[amandakatona]

Hey Clint, did you let OSCON know what we were going to write about?

[clintkitson]

Thanks for the ping @amandakatona. I just followed up and will work on it over the weekend.

[amandakatona]

@clintonskitson - from OSCON:

Hi, Clint,

That sounds like an excellent idea. I'm curious to read it myself; I've heard from various sources that a lack of persistence is something they find troubling about containers. The only caution I have here is that 500 words may be less space than you think, so be careful of including too much analysis. 500 words is a guideline, rather than a hard limit, but it's only about two pages of text in MS Word.

--Brian

[clintkitson]

Companies are turning to containers and open source as a key pillar of their infrastructure. This path is focusing on operating in new ways and EMC {code} is here to support this vision. At the heart of the strategy is embracing strategies that embrace innovation and focus on integration through open source to ensure infrastructure is more efficient and can support fast moving applications. But in order to ensure the broadest use of containers, the eco-system and platforms must now turn to integrating persistence features.

Moving from applications being installed on hosts to deploying ephemeral containers from container images promotes efficiency and scale. But when considering applications that maintain state such as databases an anti-pattern is introduced. Without the inclusion of external storage for containers, availability of the application is a problem and the hosts becomes important once again because of the data from the applications. In a world without containers or with virtual machines, the inclusion of state or data with the operating system has been an accepted norm. But today where open source infrastructure is dominated by containers, leveraging external storage to separate state from the host and container is a true enabler.

For this article we focus describing characteristic of four platforms that have enabled or are working towards enabling external storage with containers: Docker, Mesos, Kubernetes, and Cloud Foundry. Similarily, the architectures of these platforms include both container scheduling and container runtime layers that can play a role in integration.

In order to properly capture the state of the eco-system with persistence, we we think about the progress in the following areas.

• The orchestration process or integration of storage platforms to container runtimes to get external volumes available for containers and their applications
• Composeability of volume functionality being driven upwards through container runtime APIs where volumes are available as first class citizens
• The integration between storage platforms and container runtimes through sustainable in sustainable ways
• Bringing lifecycle management of volumes into container runtimes
• Applications being aware and able to take advantage of external storage

The container runtime is where the heavy lifting occurs for creating and removing containers. Important orchestration work here takes place in connecting containers to requested resources from storage platforms. Here we focus on composeability, where the correct path forward exposes volume functionality solely through the container runtime API versus a separate interface.

The integration with storage platforms is also important. This integration can differ across container runtimes and storage platforms. It can take place through the runtime having integrations built in for distinct storage platforms, or integrated in more sustainable means of a well defined public API. This API should achieve the decoupling of storage platform support from container runtime dependencies.

The next important step is to ensure the lifecycle of volumes be handled separate from containers. This means container platforms have the ability to attain more resources on their own. Similar to requesting volumes with containers this is a feature that is controlled through a runtime API, and exposed upstream to schedulers. Using this volumes can be created and removed by container consumers.

How we define an application with storage and how the application leverages storage is also important. Here an application can either be more static and seamless consumers of the new external storage, or it can be more dynamic and be able to interact with storage platform features directly. Being organic and marrying the lifecycle of volumes to the operational aspects of an applications is key to running applications that scale and is aligned to two-layer scheduling mechanisms that are being implemented.

Up to now we have painted the picture of functionality that exists today. We finish with some questions that are to be answered.

• Can we introduce lifecycle management of volumes throughout container platforms?
• How far will applications go in taking advantage of storage features?
• Can there be native integration of container runtimes with storage platforms?

The EMC {code} community is a great place to join the conversation. Help us solve for persistence with open source infrastructure. Come to our session to here more about where the eco-system fits in this picture.