tweak github actions setup so that docker layers are cached better

[jrheard]

right now docker builds take ~5 minutes, and no layers are cached except the layers from the last stage, which don't save us any time. cargo-chef and buildx's gha caching are supposed to do nice caching things, but they don't seem to be doing that in this setup right now.

one possible theory: maybe cargo-chef is violating some assumptions of buildx's caching? maybe buildx only caches the final layer intentionally for some reason? no clue.

[jrheard]

the situation is even worse now that i changed docker-compose-test to build its own rask_api image. the gha docker image layer caching story is evolving and not super great atm. we need to cache built deps/layers across these three situations:

• the docker-compose-test.yaml rask_api build stage
• the cargo test run
• the build stage for the final Dockerfile-based rask_api:latest image

right now i don't think there's working caching for any of those steps :)

[jrheard]

thought about this a bit more, here's a sketch of a possible direction:

• ditch --release entirely for now, it just complicates things
• smush the current two files into one file, use needs to have the final build step only happen if the testing/tarpaulin step succeeds
• figure out how to use dockerfile targets
• use https://github.com/docker/build-push-action/blob/master/docs/advanced/local-registry.md to build+store an up-to-date version of the rask_api image so that it can be picked up by the "start containers" step
• https://github.com/docker/buildx/pull/535 (gha buildx caching) for the build-and-push steps
• have the cargo test step happen in the context of a particular dockerfile target, rather than installing the rust toolchain and running cargo test directly in the context of the github action workflow

so in my head the main github action workflow would end up looking something like this:

• local docker registry
• set up buildx
• build rask_api target of dockerfile.test, use gha caching for image layers, push to local registry
• build testing target of dockerfile.test, use gha caching for image layers, push to local registry
• docker-compose -f docker-compose-test.yaml up -d (pulls rask_api image from local registry)
• run testing image from local registry, somehow connect it to services provided by docker-compose (???)
• somehow cause cargo-tarpaulin to happen (???)
• build production dockerfile, push to dockerhub

yeesh. that sounded simpler in my head.

[jrheard]

TODO: see what other projects do!!!