viafoura / deprecated-buri

Ansible-driven AMI generation
Apache License 2.0
40 stars 14 forks source link

Aminator Integration #13

Closed jhohertz closed 10 years ago

jhohertz commented 10 years ago

Issue moved from original repo:

While the ansible provisioner for Aminator seems to work great, the real magic is in netflixoss-ansible is in the system recipes themselves, not the hook for Aminator.

The real value in Aminator is really in terms of ability to handle high volume. By working only with volumes that can be mounted on sd[f-j][1-16], and providing locking, a whole lot of jobs can run at once. Getting it to move to partitioned volumes means only being able to mount to sd[f-j], reducing the number of nodes by a factor of 16.

As well, by allowing management of bases (typically just a java web container, but for special cases wrapping other services as well), the work per re-imaging should be minimized some.

Issue #3 discusses how we might explore targeting a non-partitioned volume as Aminator expects. The issue Netflix/aminator#129 discusses (possibly way too much on my part) how to support partitioned volumes, and tangent to that, HVM machine types from within Aminator natively.

Bakery4AWS, from @pas256 who also authors netflixoss-ansible is one possible direction, yet to be explored. It also makes use of something called Packer which may be a good piece to all this. Not sure it's state on HVM, partitions, etc.

We could also do things in terms of a bridge between buri and netflixoss-ansible in other ways.

One other note to the above. We have a chroot-oriented foundation and base build that is work-alike to Aminator, but for the concurrent run capability. (and some features).

Once the ability to deal with partition table disks is reconciled, we will need a variation on the ansible provisioner to work with the new context for working with the chroot volumes to work w/ buri. (vs. running ansible inside the chroot).

jhohertz commented 10 years ago

32 brings, what I think, will help merge these worlds. It is generating flat, no-partition AMIs of all types PVM/HVM EBS/S3 backed. Now in testing.

With this, we could generation foundations for Aminator, work with the same images at it, etc.

jhohertz commented 10 years ago

44 has the majority of the effort for this now, so I'll close it as it's scope is largely duplicated there now.