Docker and containers

[johnwatson484]

Adding Docker standards.

[johnwatson484]

I've suggested that we should make use of parent images as that is more DRY and means we can have greater control over the content of all our containers. We can also patch some issues at root level.

Within FFC we are still looking to see where these parent images should be hosted as we will have some services in AWS and some in Azure. We may end up hosting them in both Azure Container Registry (ACR) and Elastic Container Registry (ECR) or we could use another platform such as Dockerhub to host them and conversations are ongoing about where we put them.

But as we have other teams using Docker then maybe we should all be using the same parent images, rather than different flavours of the same thing everywhere. Then there is less duplicated effort in supporting. I think it's worthy of a discussion.

[Cruikshanks]

TBH, I'm struggling with this one.

I'm only just getting back on the Docker horse, so there is a lot of references to stuff I've as yet had no experience with. And I believe most devs in DDTS have a similar level of experience.

Also with all respect to @johnwatson484 this very much seems to come from an experience working on complex micro-service, multi-application services. There are quite a number of other services which are essentially an app and a database.

For example, there is a whole section on multi-stage builds with the focus on building containers for local development, unit testing, and then production. I think there is just as valid a use case for those of us who are happily developing directly on the host, and already have our tests running as part of an external CI, but just want a steer on is what the Dockerfile to create a production image should look like.

Plus the Dockerfile best practices on multi-stage builds focus on how to layer your build to reduce the size of your final image. So should the layering be the 'standard', and the build stages 'guidance' on how to support multiple environments?

So I'm struggling with what is a standard and therefore we should do, and what is guidance and is recommended we could do? And how to apply that to my own experience and services.

But so much of this is useful and shouldn't be lost that I'm starting to think we could start new Docker sections under /standards and /guidance as they'll probably be too much to contain in just 2 documents!

For example, I can see separate documents covering standards for Node and .Net production Docker containers, including example Dockerfiles with expected structure/content. Then in the guidance section the stuff on how you can use multi-stage builds to support local development, internal CI etc. Plus the stuff on 'Composing multiple repositories for local development'.

And rather than doing all of that in one go, deal with each standard and guidance document individually.

I guess TLDR, there is a lot here, but if we could break it down a bit more, I at the very least would have an easier time giving productive feedback.

[johnwatson484]

TBH, I'm struggling with this one.

I'm only just getting back on the Docker horse, so there is a lot of references to stuff I've as yet had no experience with. And I believe most devs in DDTS have a similar level of experience.

Also with all respect to @johnwatson484 this very much seems to come from an experience working on complex micro-service, multi-application services. There are quite a number of other services which are essentially an app and a database.

For example, there is a whole section on multi-stage builds with the focus on building containers for local development, unit testing, and then production. I think there is just as valid a use case for those of us who are happily developing directly on the host, and already have our tests running as part of an external CI, but just want a steer on is what the Dockerfile to create a production image should look like.

Plus the Dockerfile best practices on multi-stage builds focus on how to layer your build to reduce the size of your final image. So should the layering be the 'standard', and the build stages 'guidance' on how to support multiple environments?

So I'm struggling with what is a standard and therefore we should do, and what is guidance and is recommended we could do? And how to apply that to my own experience and services.

But so much of this is useful and shouldn't be lost that I'm starting to think we could start new Docker sections under /standards and /guidance as they'll probably be too much to contain in just 2 documents!

For example, I can see separate documents covering standards for Node and .Net production Docker containers, including example Dockerfiles with expected structure/content. Then in the guidance section the stuff on how you can use multi-stage builds to support local development, internal CI etc. Plus the stuff on 'Composing multiple repositories for local development'.

And rather than doing all of that in one go, deal with each standard and guidance document individually.

I guess TLDR, there is a lot here, but if we could break it down a bit more, I at the very least would have an easier time giving productive feedback.

Thanks Alan, some good points there.

I've clarified the use of parent images for large microservice based projects as opposed to those that are more monolithic.

I still feel a lot of the rest is still relevant for monoliths and microservices. Excluding parent images I've used the same multi stage build approach.

One of the biggest benefits of using containers is that you can run locally exactly what will be deployed to production. A multi stage build approach also allows you develop and run tests within the container so you can see production behaviour as you code. You can mount a local volume and use a file watcher so as you write code the code is mirrored in the container which saves you having to keep rebuilding the container.

As you say, you could develop outside the container and let your CI worry about whether the container works, but I don't think that's the best approach as then you've got to check the code works then check it works in a container. The tech stack used will have an impact on the difference between the local machine and the container. For example developing a .Net Core service on a windows machine may behave differently than when it is run in a Linux container. If you're developing in the container you don't need to worry about the variation.

I like your suggestion of breaking this into standards and guidance. I'll refactor this PR to meet that approach, we can then see if we need to break it down into separate PRs or if the new format is now clearer and easier to support/give feedback on.

[Cruikshanks]

I still feel a lot of the rest is still relevant for monoliths and microservices. Excluding parent images I've used the same multi stage build approach.

@johnwatson484 Having checked out this PR it finally inspired me to get off my 🍑 and have a proper look at Docker, and how I'd go about dockerising one of our ruby services. After just a few days

• 💯*💯 yes for multi-stage builds! Why wouldn't you do it in this way?!
• docker-compose override files is actually a nifty way of managing different environments and settings so I'm fully on board with that
• I now at least have enough experience to know I'm seriously underqualified to pass judgement on this yet!

So thank you for breaking it up. I still feel that has value. But feel free to quietly ignore everything else I said 😁

[johnwatson484]

Being new to Docker, I can't claim to understand the subject matter very well - a bit more context or a link to read more would be good for me. But hopefully I've done my QA bit by pointing out a couple of typos :)

Thanks, appreciated 👍

[johnwatson484]

"Note that services built using .Net Framework vs .Net Core cannot be run using Linux containers."

Implies but doesn't actually state that .Net Core can run using Linux containers. Perhaps rewrite as "Services built using .Net Framework cannot run using Linux containers, but services built building using .Net Core can. Therefore..."

Good point, updated as suggested.

[johnwatson484]

This is more of a question really... So we seem to have Container standards but Docker guidance. I can see that the main Container standard is "use Docker" which goes some way (all the way???) to explaining that but is there actually no Container guidance or Docker standards (that couldn't be applied to all containers?). Just a bit confusing, especially for anyone coming to this new. But may be a red herring, not sure.

Originally it was called "Docker guidance" and "Docker standards". I changed the standards as I felt that the principles are applicable to any container technology and if there are teams who have a valid use case for not using Docker then they should still abide by the standards in the document. (Though I can't imagine what that use case would be).

Whereas the guidance is more specific to using Docker as opposed to containers in general.

[johnwatson484]

Limited experience in Docker here as well but I think this looks good – nice and easy to scan through, especially the standards. I have a few very small tweaks to suggest but that's it.

Many thanks, I've committed your suggested changes 👍