Open gaocegege opened 2 years ago
If we move to the frontend design, we can integrate with buildkit ecosystem seamlessly. For example, we can use buildx action in GitHub:
name: Name
on:
push:
branches: [dev]
jobs:
push_to_registry:
name: Push Docker image to GitHub Packages
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v2
- name: Set up QEMU
uses: docker/setup-qemu-action@v1
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v1
- name: Push to GitHub Packages
uses: docker/login-action@v1
with:
registry: ghcr.io
username: ${{ github.repository_owner }}
password: ${{ secrets.CR_PAT }}
- name: Build and push
id: docker_build
uses: docker/build-push-action@v2.0.1
with:
context: ./app/
push: true
file: ./app/build.envd
build-args: |
...
Ref #510
Buildkit frontend requires a image from us, just like docker/dockerfile
.
The image docker/dockerfile
contains a CLI binary:
package main
func main() {
if err := grpcclient.RunFromEnvironment(appcontext.Context(), dockerfile.Build); err != nil {
logrus.Errorf("fatal error: %+v", err)
panic(err)
}
}
It wraps the dockerfile.Build
with grpcclient.RunFromEnvironment
. Function dockerfile.Build
is the build func for Dockerfile.
We can provide a similar implementation in tensorchord/envd-frontend
image if we make envd a frontend of buildkit.
package main
func main() {
if err := grpcclient.RunFromEnvironment(appcontext.Context(), envd.Build); err != nil {
logrus.Errorf("fatal error: %+v", err)
panic(err)
}
}
And the build.envd can be:
syntax=tensorchord/envd-frontend:<version>
We can use buildctl
to build envd images with the help of the frontend:
buildctl build \
--frontend gateway.v0 \
--opt source=build.envd \
--opt context=.
The gateway in buildkit will forward requests to our own tensorchord/envd-frontend
, and transform frontend starlark statements to LLB.
After #606 is merged, we already have the build func. But we cannot make envd a frontend of buildkit:
LocalDirs
in SolveOpt
to expose cache dir to the build process. The LocalDirs
is not supported in build func. We should make cache dir a llb.Local
, then mount it into the llb states. See https://github.com/r2d4/mockerfile/blob/master/pkg/build/build.go#L83containerimage.config
and name
in SolveOpt
. SolveOpt
is not exposed in the gateway client. We should use ResolveImageConfig
in the gateway client to re-implement the logic.We rely on LocalDirs in SolveOpt to expose cache dir to the build process. The LocalDirs is not supported in build func. We should make cache dir a llb.Local, then mount it into the llb states.
It seems we could leverage named context https://www.docker.com/blog/dockerfiles-now-support-multiple-build-contexts/ if we made envd a buildkit frontend, but we need to pass an extra argument to docker buildx build
, like:
docker buildx build --build-context local-cache=~/.cache/envd ...
The cache dir should be added by default in envd
binary, Thus I think we could just make it in build func instead of a extra arg. :thinking:
Please correct me if I missing something 🤔, we cannot access dir outside the build context dir during build time if envd is used as a buildkit frontend since buildx only adds the context dirs to LocalDirs
https://github.com/docker/buildx/blob/701c548e46348da2958104907c9572ea7ce6ab52/build/build.go#L1376
Yep, we cannot access dirs if it is not added into buildkit as a build context.
I mean we should keep localdir in LocalDirs by default in envd
's binary. If users use buildctl
, they can use --build-context
to add cache dir as build context.
BTW, if you are interested in buildkit integration with envd, maybe you can have a look at #124
We are working on supporting debug functionality in envd, similar to docker buildx. But there are many challenges especially from buildkit.
Mocker uses github.com/moby/buildkit/gateway/client instead of github.com/moby/buildkit/client.
It introduces a new abstraction Build func https://github.com/r2d4/mockerfile/blob/140c6a912bbfdae220febe59ab535ef0acba0e1f/pkg/build/build.go#L37
The benefit is that you can do some post-solve logic like this https://github.com/r2d4/mockerfile/blob/140c6a912bbfdae220febe59ab535ef0acba0e1f/pkg/build/build.go#L65