RFE: Layered container images using Docker Builder

[ghost]

This issue was originally opened by @maxking as hashicorp/packer#6000. It was migrated here as a result of the Packer plugin split. The original body of the issue is below.

---

This feature request follows from hashicorp/packer#5972 for allowing one layer per-provisioner in docker builder, so that we have some opportunity to optimize the container image for sharing layers.

I spent some time hacking on trying to implement a suggestion from @mwhooker but hit a snag when I realized that just changing the HookedProvisioner to something that commits after each provisioner wouldn't work and it would instead result in multiple images, produced after each provisioner ran and sharing no layers.

If I understand correctly, packer runs the base image, provisions it and then commits to an image. To achieve layering, I assume that we'd have to run a provisioner, commit the image and then teardown the entire thing and re-do the StepRun and StepConnect using the image committed in the previous step.

I just wanted to open this issue to discuss what you'd is the best way to achieve this, before I go around making too many intrusive changes?

[michellelynne]

I would also like one layer per provisioner, since I am running into the "layers too large at 10GB" where the push will fail. I'm wondering why this closed and if there is another ticket I should be following? My work around is to string together multiple builds.

[colearendt]

This would be super useful to improve build time for docker images while iterating on a build. In particular, docker can often be a useful "dev" target for other builders (AMIs, etc.). In that context, where a build is happening on a laptop / etc., caching can be a game changer for speed / cycle time on iterations.