Non-ECS Cluster vs ECS Cluster

[hapnermw]

Elsewhere in the docs there are brief instructions for creating a CoreOS cluster on AWS ECS.

No information is provided contrasting Non-ECS vs ECS CoreOS clusters. No mention of the fact that ECS's provisioning of containers (tasks) duplicates fleet's container provisioning is made and when to use which.The fact that ECS container provisioning is mostly useless for real work isn't mentioned.

For CoreOS users to successfully deploy and manage a production cluster on CoreOS supported 'platforms' they must understand, in detail, how to combine 'platform' management functionality with CoreOS management.

The level of info you currently supply for AWS EC2 is barely enough to get a test cluster up and running. It is way below the bar for what is required to put a cluster into production on EC2.

[benmccann]

+1 I'm looking at the docs and trying to understand why there's a section about running with CoreOS and ECS because it doesn't make sense to me why you'd combine the two.

[robszumski]

You are both correct, this document could be improved a bit. Here are a few thoughts on why CoreOS makes a great base for ECS:

• base OS receives frequent security enhancements and bug fixes
• runs the latest versions of Docker without you having to manage it yourself
• easy to run a single machine dev/staging env on your laptop with Vagrant. Should function identical in terms of container execution, kernel features, etc. Obviously you'd have to launch the containers differently.
• the OS serves as a common compatibility layer. It's designed to run the same on EC2, Google Compute Engine, etc. Makes it easy to switch providers or bridge clouds.

[benmccann]

Yes, but those are all reasons CoreOS is great by itself :-) I still don't know why I'd want to run the ECS agent on it instead of just running CoreOS alone...

[hapnermw]

This quote from the AWS ECS page tells it all ...

‘Amazon EC2 Container Service (ECS) is a highly scalable, high performance container management service that supports Docker containers and allows you to easily run applications on a managed cluster of Amazon EC2 http://aws.amazon.com/ec2/ instances. Amazon ECS eliminates the need for you to install, operate, and scale your own cluster management infrastructure.’

While ECS currently sucks as a hosted cluster service, AWS expects it to be THE form of cluster compute its customers use. It is clearly meant to replace any and all other contenders. Just as Dynamo is to replace any and all other NoSQL contenders.

Given the current lack of ECS functionality, about all it is good for is to automate the provisioning of CoreOS machines - but this will likely change.

Since AWS is unlikely to partner with CoreOS for its hosted cluster functionality, CoreOS needs to think seriously about how to out compete it. One way to do this is to offer a cluster provisioning and management service - one that automates the provisioning and management of cluster machines across AWS, Azure, Google Compute, VMWare, etc. Simply providing CoreOS images for these environments isn’t enough.

I view the current CoreOS docs on ECS use as a somewhat misguided, short term attempt to grapple with this larger issue.

On Sat, Apr 18, 2015 at 2:46 PM, Ben McCann notifications@github.com wrote:

Yes, but those are all reasons CoreOS is great by itself :-) I still don't know why I'd want to run the ECS agent on it instead of just running CoreOS alone...

— Reply to this email directly or view it on GitHub https://github.com/coreos/docs/issues/454#issuecomment-94203637.

[bobzoller]

I've got a specific use case. remind101/empire is a control layer that leverages ECS to provide a Heroku-like workflow. In fact it implements a subset of the Heroku Platform API, so in some cases you can still use the Heroku Toolbelt CLI.

Empire is the shortest path (not to mention a good one) to moving our engineering teams off of Heroku. At the same time, I've got desires in the Delivery Engineering world to move our shared infrastructure onto CoreOS... but for those loads it makes sense to use fleet directly.

In the long run, maybe Empire will support fleet as a scheduler, or maybe we'll decide to pursue deis or flynn or some other 12-factor-app-oriented platform that "speaks CoreOS" already. But for now, I'm considering using CoreOS and the ECS agent in concert.

This issue isn't exactly the place for it (github message?), but I'd be thrilled to get feedback if folks have it.