genuinetools / img

Standalone, daemon-less, unprivileged Dockerfile and OCI compatible container image builder.
https://blog.jessfraz.com/post/building-container-images-securely-on-kubernetes/
MIT License
3.9k stars 231 forks source link

Use credential-helper creds, bypass prompt. #276

Closed champloo11 closed 4 years ago

champloo11 commented 4 years ago

I was looking at what it would take to solve respecting the helpers for GCR and ECR for https://github.com/genuinetools/img/issues/128 especially within the context of an automated build environment.

Turns out it's basically already implemented, we just need to use both the user (or access token) and secret from the authConfig that is already being populated by the helper.

This change bypasses the username and password prompt if none are passed in, and we already get them from the credential helper-- automating gcr and ecr creds helper authentication for img

Tested against my own private repos in docker-credential-gcloud (via gcloud auth and docker-credential-ecr).

champloo11 commented 4 years ago

I'm not sure why (perhaps dependencies that have changed) since that last merge two months ago, but I think the ./contrib/e2e-dockerfiles-build-test.sh test may be broken, hence why the Travis CI build is failing.

git checkout e16396e00ce71a7184a214da9f473daa65ef83a6 # the last commit before this branch
make
sudo make install
./contrib/e2e-dockerfiles-build-test.sh

The above successfully makes and installs, but fails on the following:

./contrib/e2e-dockerfiles-build-test.sh
Cloning into '/tmp/tmp.mZwW4LPTWN'...
remote: Enumerating objects: 532, done.
remote: Counting objects: 100% (532/532), done.
remote: Compressing objects: 100% (363/363), done.
remote: Total 532 (delta 37), reused 309 (delta 14), pack-reused 0
Receiving objects: 100% (532/532), 2.58 MiB | 3.92 MiB/s, done.
Resolving deltas: 100% (37/37), done.
Running in parallel with 1 jobs.
Academic tradition requires you to cite works you base your article on.
When using programs that use GNU Parallel to process data for publication
please cite:

  O. Tange (2011): GNU Parallel - The Command-Line Power Tool,
  ;login: The USENIX Magazine, February 2011:42-47.

This helps funding further development; AND IT WON'T COST YOU A CENT.
If you pay 10000 EUR you should feel free to use GNU Parallel without citing.

To silence this citation notice: run 'parallel --citation'.

/home/daymon/img/contrib/e2e-dockerfiles-build-test.sh dofile lpass/Dockerfile
Building r.j3ss.co/lpass:latest for context lpass
Building r.j3ss.co/lpass:latest
Setting up the rootfs... this may take a bit.
[+] Building 0.1s (2/2) FINISHED                                                                                       
 => [internal] load .dockerignore                                                                                 0.1s
 => => transferring context: 2B                                                                                   0.0s
 => [internal] load build definition from Dockerfile                                                              0.1s
 => => transferring dockerfile: 2B                                                                                0.0s
Error: failed to solve: failed to read dockerfile: open /tmp/buildkit-mount080956811/Dockerfile: no such file or directory

If anyone else has a better insight into the build pipeline a :+1: or :-1: on whether this seems to be correct would be awesome.

AkihiroSuda commented 4 years ago

revendoring buildkit (master) may fix the issue

champloo11 commented 4 years ago

The next version of buildkit that is available upgrades us from 0.5 to 0.6, and there are breaking interface and provider changes.

Would it be possible to run the Travis CI pipeline on master to absolutely confirm that e2e tests are broken in the build?

If we can confirm that master's e2e is broken, then we can create a ticket to fix the build and I can take a look at what that might take.

AkihiroSuda commented 4 years ago

restarted e2e CI on master: https://travis-ci.org/genuinetools/img/jobs/613801328

champloo11 commented 4 years ago

Mind rerunning the full pipeline? The master build failed but failed to return an error code (I can see about adding a check for that later)

https://travis-ci.org/genuinetools/img/jobs/613801328?utm_medium=notification&utm_source=github_status image

I think because running that one stage in isolation meant the binary from the make install wasn't present

codecov-io commented 4 years ago

Codecov Report

Merging #276 into master will not change coverage. The diff coverage is 0%.

Impacted file tree graph

@@          Coverage Diff          @@
##           master   #276   +/-   ##
=====================================
  Coverage       0%     0%           
=====================================
  Files          16     16           
  Lines         984    986    +2     
=====================================
- Misses        984    986    +2
Impacted Files Coverage Δ
main.go 0% <0%> (ø) :arrow_up:
login.go 0% <0%> (ø) :arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more Δ = absolute <relative> (impact), ø = not affected, ? = missing data Powered by Codecov. Last update e16396e...0bbe714. Read the comment docs.

AkihiroSuda commented 4 years ago

restarted entire CI on master https://travis-ci.org/genuinetools/img/builds/636221063

champloo11 commented 4 years ago

@AkihiroSuda I think that particular error has been occurring for a while, here is a build on master 6 months ago:

https://travis-ci.org/genuinetools/img/jobs/597047115

There are two potential problems:

Should we create a ticket and merge (now that the pipeline has accepted the PR), or block this fix until we can get to the bottom of fixing the pipeline?

champloo11 commented 4 years ago

^^ Re: just in case it got missed in the noise.

codecov-commenter commented 6 days ago

:warning: Please install the 'codecov app svg image' to ensure uploads and comments are reliably processed by Codecov.

Codecov Report

Attention: Patch coverage is 0% with 3 lines in your changes missing coverage. Please review.

Project coverage is 0.00%. Comparing base (e16396e) to head (0bbe714). Report is 30 commits behind head on master.

Files with missing lines Patch % Lines
login.go 0.00% 2 Missing :warning:
main.go 0.00% 1 Missing :warning:

:exclamation: Your organization needs to install the Codecov GitHub app to enable full functionality.

Additional details and impacted files ```diff @@ Coverage Diff @@ ## master #276 +/- ## ====================================== Coverage 0.00% 0.00% ====================================== Files 16 16 Lines 984 986 +2 ====================================== - Misses 984 986 +2 ```

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.