ReconfigureIO / platform

The backend of app.reconfigure.io

1 stars 1 forks source link

Migrate to go modules and docker with buildkit caching #310

Closed pwaller closed 5 years ago

pwaller commented 5 years ago

Headlines:

Now we use Go modules. We can drop Glide.
Using buildkit, we can allow Go to use its caches during docker build. This means that if you change code, you can get a rebuild in a matter of seconds, whilst still being functionally equivalent to a complete rebuild.

Checklist

[x] Added to RELEASE.md?
[x] Put version number in binaries. Dockerfile
[ ] Put version number in binaries. Makefile
[ ] Put version number in binaries. Jenkinsfile
[x] Drop old glide stuff.
[ ] Come up with a way of running tests in Docker.
[ ] Get tests running on Jenkins.
[ ] Upgrade docker on Jenkins (18.09 or later preferably)
[x] Upgrade golang on Jenkins (1.11)
[ ] Set DOCKER_BUILDKIT=1 when building.
[ ] Remove asset manipulation from Makefile, replace with COPY in Dockerfile into runtime image.
[ ] Scrap fresh. Come up with alternate development process.
[ ] Drop vendor directory (all references to it).
[ ] Test on staging.
[ ] Deploy.

pwaller commented 5 years ago

So far this is a proof of concept which I'm now handing over to @CampGareth to complete.

CampGareth commented 5 years ago

[ ] Added to RELEASE.md

Unsure if this should be, I think of RELEASE.md as being user-facing changes only. If it's dev-facing too then this can be included

pwaller commented 5 years ago

Please consider https://github.com/ReconfigureIO/eTeak/blob/8e17fa5392667aa69dec47bf910c3032aa3d143a/RELEASE.md as a model. We mark non-user facing changes with [internal]. I see no reason not to do this. I think it's good to be in the habit of always documenting what changes you're making.

CampGareth commented 5 years ago

Note to self: Jenkins stuff for this is hard so tackle it last.

CampGareth commented 5 years ago

Oh dear, docker-compose doesn't currently support buildkit: https://github.com/docker/compose/issues/883#issuecomment-440795724

That takes this from modifying our existing docker-compose file and workflow to coming up with a new one. I'm okay with that but it looks like a rabbit hole so I'm going to leave it for now.

pwaller commented 5 years ago

This should be straightforward. Just omit the build directives in docker compose, and do adocker build followed by docker compose up as the default make rule.

Essentially, the semantics of the docker-compose file will be "It expects the image to already exist". At least, until compose has buildkit support.

CampGareth commented 5 years ago

This would've sped up testing within docker and would've been nice to have, however development is spinning down so closing this PR.