Feature request: allow skaffold build once and tag&push Image to multiple repos

[dkirrane]

I'd like to use Skaffold+Jib to build my container image and push it to both an in-house Nexus Docker Registry and also push to eu.gcr.io

Currently I'm having to rebuild the Image for each repository using skaffold profiles. Could Jib build the image once and tag it with each repo name I need (same tag version) and then push it to each repo.

Benefit: Build once, push to different repos. Then use skaffold profiles to do the deployment to the different environments.

[briandealwis]

Unfortunately this isn't possible. Basically Skaffold+Jib ends up running something like

mvn prepare-package jib:build -Dimage=<skaffold-image>

Maven supports invoking plugin goals with different <execution> IDs, so I wondered if you could use the args field to add additional jib:build to that command-line (e.g., jib:build@id2). Unfortunately Jib treats command-line properties as over-riding those set in the pom, so the -Dimage=xxx will prevail.

Instead of Skaffold profiles, you could duplicate your artifact in your skaffold.yaml and just never reference the second artifact. For example:

build:
  artifacts:
  - image: gcr.io/project/image
    context: my-jib-project
    jibMaven: {}
  - image: local.nexus.host/project/image
    context: my-jib-project
    jibMaven: {}

Skaffold will warn:

WARN[0015] image [localhost:5000/foo] is not used by the deployment 

[dkirrane]

@briandealwis I tried with duplicate artifacts but only 1 image foo is passed to Maven:

$ VERSION=1.2.3 skaffold run -v debug 2>&1 | grep -i 'Running command'
"Running command: [mvn -Djib.console=plain --non-recursive prepare-package jib:build -Dimage=eu.gcr.io/project/foo:1.2.3]"

apiVersion: skaffold/v1beta10
kind: Config

build:
  tagPolicy:
    envTemplate:
      template: "{{.IMAGE_NAME}}:{{.VERSION}}"
  artifacts:
    - image: eu.gcr.io/project/foo
      context: ./app
      jibMaven: {}
    - image: local.nexus.host/bar
      context: ./app
      jibMaven: {}

[briandealwis]

Hmm it worked for me with skaffold dev

[briandealwis]

skaffold run from examples/jib works fine for me with the following change:

--- a/examples/jib/skaffold.yaml
+++ b/examples/jib/skaffold.yaml
@@ -4,6 +4,8 @@ build:
   artifacts:
   - image: gcr.io/k8s-skaffold/skaffold-jib
     jibMaven: {}
+  - image: localhost:5000/example
+    jibMaven: {}

 # optional profile to run the jib build on Google Cloud Build
 profiles:

Note that grepping for 'Running' as you're showing above will mask any build failures as I learned when trying to reproduce :-)

$ skaffold run -v debug 2>&1 | grep -i running.*mvn
time="2019-05-24T22:07:27-04:00" level=debug msg="Running command: [/Users/bsd/go/src/github.com/GoogleContainerTools/skaffold/examples/jib/mvnw -Djib.console=plain --non-recursive prepare-package jib:dockerBuild -Dimage=gcr.io/k8s-skaffold/skaffold-jib:v0.29.0-144-g90059f1e-dirty]"
time="2019-05-24T22:07:32-04:00" level=debug msg="Running command: [/Users/bsd/go/src/github.com/GoogleContainerTools/skaffold/examples/jib/mvnw -Djib.console=plain --non-recursive prepare-package jib:dockerBuild -Dimage=localhost:5000/example:v0.29.0-144-g90059f1e-dirty]"

And I see similar things when I build against a remote cluster with jib:build.

[balopat]

The "build once & deploy multiple places" is an interesting use case for CI/CD. We could implement it in a way that skaffold deploy / tagging / repushing is a separate phase. If people are interested in this, please thumbs up.

[balopat]

Also related to https://github.com/GoogleContainerTools/skaffold/issues/1269

[tstromberg]

Adding an update because this has so many upvotes: I believe @nkubala tried to implement this a few weeks ago, but found it difficult to complete. I'll let him comment with the details.

[nkubala]

I took a stab at addressing both this and #1269 in one go, which in retrospect was too much at once. for this issue the implementation would be a lot simpler if we just treated retagging/repushing as a separate phase, since builders like jib and kaniko build directly to the registry.

deploying to multiple clusters is a different beast entirely and i think out of scope for this issue.

[aaron-prindle]

Friendly ping, any update here on the feasability of this after refactors or #1269 ? @nkubala does it make sense to re-prioritize this due to the # of reactions? I see this was moved to the Icebox [P2+], wanted to see if it might be worth looking at again in upcoming milestones

[nkubala]

@aaron-prindle I'll try and get to this one in the next few sprints

[afirth]

two more use cases:

• ECR lifecycle policies rely on tag-prefixes. Therefore, being able to tag for example v0.0.1 and also prod-v0.0.1 would be useful
• some systems (e.g. airflow) spawn workers. It's much easier to set the worker tag to prod than try to keep it up to date, particularly as kustomize variable expansion of the image seems to happen before skaffold edit set image

[briandealwis]

@afirth could you expand on your second point? Are you using Skaffold to deploy using kustomize? Would it make sense to use skaffold render for your case?

[afirth]

@briandealwis turns out the second one is moot in this particular case of airflow - it is smart enough to use it's own image tag when spawning workers, but here's the scenario:

1. in kustomize create a variable (pseudo) $image = .spec.containers.[0].image
2. use that variable in a manifest e.g. .spec.template.spec.containers.[0].env.IMAGE = $image
3. invoke skaffold render expected the env is set to the final image tag observed the env is set to the placeholder image tag

I suspect that fixing this is fairly involved and unless someone else has a compelling use case I'd just ignore it. There are workarounds that I think would be feasible too, for example if the final tag is available or exported as an env var kustomize can pick that up. Happy to provide a minimal repro if needed, if so just open another ticket and ping me. The ECR one is much more frustrating - I don't have a workaround for that

[jessequinn]

Any update on the multi-registry retagging?