Redo Dockerfile - Githubissues

jordemort commented 2 years ago

Build the currently checked-out code, instead of cloning the repo inside the Dockerfile. This makes it much easier to build a container for a particular branch or commit; people working on personal forks will be able to build containers for their forks without modifying the Dockerfile.
Switch from Alpine to distroless; I couldn't actually get the current version of the Dockerfile to build, it kept dying with some error about gvisor. Aside from building with no trouble, the new Dockerfile reduces the size of the image from 23MB to 9MB.
Move Dockerfile into the root; this is a matter of taste, but allows one to simply docker build the directory instead of having to also specify the path to the Dockerfile. As part of this, I removed the config and Makefile from the docker directory, since they seemed specific to someone's setup and incomplete without that context.

Happy to add on a workflow to automatically build and publish the container, if you're interested.

pufferffish commented 2 years ago

Happy to add on a workflow to automatically build and publish the container, if you're interested.

If you don't mind that would be great. Are you going to open a new PR for this?

jordemort commented 2 years ago

If you don't mind that would be great. Are you going to open a new PR for this?

What's your preference? I could do that or I could add it on here.

pufferffish commented 2 years ago

If you don't mind that would be great. Are you going to open a new PR for this?

What's your preference? I could do that or I could add it on here.

I prefer adding it on here. Thanks!

jordemort commented 2 years ago

Started on the workflow but I think I need you to approve it. Not sure if we can publish to GitHub's built-in container registry without generating an access token; you might need to set something up in the repository secrets if this doesn't work.

jordemort commented 2 years ago

The workflow should be ready to go. No additional setup should be required on your part, it will use the GitHub token automatically generated for the Actions run and push the container to the GitHub Container Registry; you should see a "Packages" section pop up on the side of the repo like it has on mine after the workflow has run once: https://github.com/jordemort/wireproxy

This workflow builds a multi-platform manifest for all the platforms that Google publishes a distroless base image for (really the only one of note that's missing there is 32-bit x86)

It will run on pull requests and on pushes to master. When it is run on pull requests, it will give the resulting image a tag based on the name of the pull request branch. When it is run on master, it will give the resulting image the tag latest.

You will be able to pull the resulting images form ghcr.io/octeep/wireproxy:$TAG_NAME

jordemort commented 2 years ago

I tested the workflow in this PR if you want to have a look: https://github.com/jordemort/wireproxy/pull/1

Here are the builds: https://github.com/jordemort/wireproxy/actions/workflows/container.yml

pufferffish commented 2 years ago

Odd, workflow failed because "ERROR: unexpected status: 403 Forbidden". Would you happen to know why?

jordemort commented 2 years ago

I wonder if because it's a pull request and under my user, it's denying push access. Could you try re-running the job?

jordemort commented 2 years ago

@octeep Hm, that didn't work; can you check the repository settings? I think this might be the solution: https://github.com/docker/build-push-action/issues/463#issuecomment-1125443655

pufferffish commented 2 years ago

@octeep Hm, that didn't work; can you check the repository settings? I think this might be the solution: docker/build-push-action#463 (comment)

I've already enabled it. It seems like using GITHUB_TOKEN doesn't work for PRs? See: https://github.com/actions/runner/issues/1039#issuecomment-1019972535

jordemort commented 2 years ago

@octeep Hmmm, it worked on a PR on my repo; I also didn't have to push an initial image first or do any other permission tweaks there. I had already pushed other images for other purposes though, and I'm on a paid plan, and I'm an ex-GitHub employee, so there's a few different opportunities for my account to be weird/different from yours. I'll see what I can figure out.

jordemort commented 2 years ago

I'm trying to give the GITHUB_TOKEN write permissions to packages but it doesn't seem to be happening. In the latest run, you can find this under the "Setting up job" section:

GITHUB_TOKEN Permissions
  Metadata: read
  Packages: read

After this next run fails, could you try re-running the build to see if it affects the permissions now?

The other alternative is to give up on pushing containers for pull requests, and move the push off to a separate step that is skipped on a PR (or that unconditionally succeeds even if the push fails)

jordemort commented 2 years ago

Sorry this is such a huge pain, I swear it worked without any trouble on my account, I wonder what's different...

pufferffish commented 2 years ago

Sorry this is such a huge pain, I swear it worked without any trouble on my account, I wonder what's different...

Don't worry about it. I should be thanking you for helping with docker setup since I have virtually no experience with it. I'll try your suggestion and re-run the job.

jordemort commented 2 years ago

I've started a discussion to see if anyone else can help us out here: https://github.com/orgs/community/discussions/30300

jordemort commented 2 years ago

Another thought; this seems to be broken mostly because of the permissions that are given to GITHUB_TOKEN.

Instead of using the automatically generated GITHUB_TOKEN, you could generate a personal access token with packages:write permission and make that a repository secret; we could log in with that instead. This seems a little bit icky re: permissions though and is definitely not GitHub's recommended solution.

Another idea: You could sign up for an account on Docker Hub (or Quay or Harbor if you prefer) and put the credentials in a repo secret, and we could publish there instead. This is actually my first attempt trying to automatically publish to ghcr.io; all my personal repos that build containers push them out to Docker Hub instead.

pufferffish commented 2 years ago

Another thought; this seems to be broken mostly because of the permissions that are given to GITHUB_TOKEN.

Instead of using the automatically generated GITHUB_TOKEN, you could generate a personal access token with packages:write permission and make that a repository secret; we could log in with that instead. This seems a little bit icky re: permissions though and is definitely not GitHub's recommended solution.

Another idea: You could sign up for an account on Docker Hub (or Quay or Harbor if you prefer) and put the credentials in a repo secret, and we could publish there instead. This is actually my first attempt trying to automatically publish to ghcr.io; all my personal repos that build containers push them out to Docker Hub instead.

Sure, I'll look into docker hub tomorrow morning. It's 11pm here so I'll take a break.

pufferffish commented 2 years ago

Another thought; this seems to be broken mostly because of the permissions that are given to GITHUB_TOKEN.

Instead of using the automatically generated GITHUB_TOKEN, you could generate a personal access token with packages:write permission and make that a repository secret; we could log in with that instead. This seems a little bit icky re: permissions though and is definitely not GitHub's recommended solution.

Another idea: You could sign up for an account on Docker Hub (or Quay or Harbor if you prefer) and put the credentials in a repo secret, and we could publish there instead. This is actually my first attempt trying to automatically publish to ghcr.io; all my personal repos that build containers push them out to Docker Hub instead.

I've made a docker hub account. Let's try publishing there if publishing to GitHub still doesn't work.

jordemort commented 2 years ago

Ok, you'll want to create a token over there and then put it into a repository secret here. After that, we'll want to change the workflow.

We'll need to replace CONTAINER_NAME:

    env:
      CONTAINER_NAME: docker.io/YOUR_DOCKER_HUB_USERNAME/wireproxy

...and we'll need to switch to logging into Docker Hub instead of GitHub Container Registry

      - name: Login to Docker Hub
        uses: docker/login-action@v2
        with:
          username: YOUR_DOCKER_HUB_USERNAME
          password: ${{ secrets.THE_NAME_OF_THE_SECRET_YOU_CREATED }}

pufferffish commented 2 years ago

Ok, you'll want to create a token over there and then put it into a repository secret here. After that, we'll want to change the workflow.

We'll need to replace CONTAINER_NAME:
    env:
      CONTAINER_NAME: docker.io/YOUR_DOCKER_HUB_USERNAME/wireproxy
...and we'll need to switch to logging into Docker Hub instead of GitHub Container Registry
      - name: Login to Docker Hub
        uses: docker/login-action@v2
        with:
          username: YOUR_DOCKER_HUB_USERNAME
          password: ${{ secrets.THE_NAME_OF_THE_SECRET_YOU_CREATED }}

Right, I've configured ${{ secrets.DOCKER_HUB_USERNAME }} and ${{ secrets.DOCKER_HUB_ACCESS_TOKEN }}.

jordemort commented 2 years ago

Hm, not sure if I got that wrong or if it's not allowing the secrets because I'm not the repo owner. Take a look and see if I typed things right, and maybe try a rebuild to see if it gives the secrets for you.

pufferffish commented 2 years ago

Hm, not sure if I got that wrong or if it's not allowing the secrets because I'm not the repo owner. Take a look and see if I typed things right, and maybe try a rebuild to see if it gives the secrets for you.

Apparently, secrets are not passed to workflows that are triggered by a pull request from a fork. I think we should probably just give up on pushing containers for PRs. I'll try testing your commits in a separate branch. If it works then I'll merge this PR into master.

jordemort commented 2 years ago

Apparently, secrets are not passed to workflows that are triggered by a pull request from a fork.

Ah yes, that makes sense.

I think we should probably just give up on pushing containers for PRs.

Probably for the best at this point; this workflow was based on one I use in private repos that don't get any forks. That probably also offers you the option to go back to ghcr.io if you'd rather keep everything in one place. You might still prefer Docker Hub for discoverability reasons though; I see there are a few bootleg wireproxy containers up there already.

It would still be nice if we could build the container for pull requests, even if we don't push it. It does take quite a long time to build for all the platforms, though, but there's also value in ensuring that it actually builds on all platforms, kind of a tough call there. The GitHub Actions cache should kick in once things are merged, which should help. Maybe doing a RUN go mod download before the build would create another layer that would also help with caching there. Making it a matrix build could also help but complicates assembling the various bits back into a multi-platform manifest at the end.

I'll try testing your commits in a separate branch. If it works then I'll merge this PR into master.

Ping me if you need help with anything, and thanks for wireproxy; I think it has some really interesting potential use cases for connecting together stuff running in different clouds and k8s clusters and things where you might not want or be able to be root. Using it the way I am, it's kind of like a better self-hosted ngrok.

pufferffish / wireproxy

Redo Dockerfile #45