search

GoogleContainerTools / skaffold

Easy and Repeatable Kubernetes Development
https://skaffold.dev/
Apache License 2.0
14.99k stars 1.62k forks source link

Not using local image (with helm) #3497

Closed vitorcamachoo closed 4 years ago

vitorcamachoo commented 4 years ago

Expected behavior

When using skaffold with helm, running skaffold dev it should use the latest local image built.

Actual behavior

When running skaffold dev it is building the image, but trying to use some remote image giving the following error:

Normal   Scheduled  60s                default-scheduler        Successfully assigned default/core-deployment-7cb45d7865-484pn to docker-desktop
  Normal   BackOff    26s (x2 over 55s)  kubelet, docker-desktop  Back-off pulling image "core"
  Warning  Failed     26s (x2 over 55s)  kubelet, docker-desktop  Error: ImagePullBackOff
  Normal   Pulling    12s (x3 over 58s)  kubelet, docker-desktop  Pulling image "core"
  Warning  Failed     10s (x3 over 56s)  kubelet, docker-desktop  Failed to pull image "core": rpc error: code = Unknown desc = Error response from daemon: pull access denied for core, repository does not exist or may require 'docker login': denied: requested access to the resource is denied
  Warning  Failed     10s (x3 over 56s)  kubelet, docker-desktop  Error: ErrImagePull

Information

apiVersion: skaffold/v1
kind: Config
build:
  artifacts:
    - image: core
      context: applications/core
      docker:
        dockerfile: Dockerfile
      sync:
        infer:
          - "**/*"

deploy:
   helm:
     releases:
       - name: cluster
         chartPath: .helm

values.yaml

replicaCount: 1

application:
  client: asseco
  name: cluster
  services:
    core:
      name: core
      image: core
      enabled:

application.yaml

---
{{- range $key, $val := .Values.application.services -}}
{{- if $val.enabled -}}
apiVersion: apps/v1
kind: Deployment
metadata:
    name: {{ $val.name }}-deployment
    release: {{ $.Name }}
spec:
  replicas: 1
  selector:
    matchLabels:
      component: {{ $val.name }}
  template:
    metadata:
      labels:
        component: {{ $val.name }}
    spec:
      containers:
        - name: {{ $val.name }}
          image: {{ $val.image }}
          ports:
            - containerPort: 3000

---
apiVersion: v1
kind: Service
metadata:
  name: {{ $key }}-service
spec:
  type: ClusterIP
  selector:
    component: {{ $key }}
  ports:
    - port: 3000
      targetPort: 3000
---
{{- end -}}
{{- end -}}
balopat commented 4 years ago

Thanks for opening @vitorcamachoo! Can you share the output of your logs when running with skaffold dev -v debug?

vitorcamachoo commented 4 years ago

@balopat Hii, here is the output:

INFO[0000] starting gRPC server on port 50051           
INFO[0000] starting gRPC HTTP server on port 50052      
INFO[0000] Skaffold &{Version:v1.1.0 ConfigVersion:skaffold/v2alpha1 GitVersion: GitCommit:2f14d99fc5f81e3a52dd76d43cad8d014f150327 GitTreeState:clean BuildDate:2019-12-21T13:45:01Z GoVersion:go1.13.5 Compiler:gc Platform:darwin/amd64} 
DEBU[0000] config version (skaffold/v1) out of date: upgrading to latest (skaffold/v2alpha1) 
INFO[0000] applying profile: helm                       
DEBU[0000] overlaying profile on config for field Build 
DEBU[0000] overlaying profile on config for field artifacts 
DEBU[0000] overlaying profile on config for field insecureRegistries 
DEBU[0000] overlaying profile on config for field tagPolicy 
INFO[0000] no values found in profile for field TagPolicy, using original config values 
DEBU[0000] overlaying profile on config for field BuildType 
INFO[0000] no values found in profile for field BuildType, using original config values 
DEBU[0000] overlaying profile on config for field Test  
DEBU[0000] overlaying profile on config for field Deploy 
DEBU[0000] overlaying profile on config for field DeployType 
DEBU[0000] overlaying profile on config for field statusCheckDeadlineSeconds 
DEBU[0000] overlaying profile on config for field kubeContext 
DEBU[0000] overlaying profile on config for field PortForward 
DEBU[0000] Defaulting build type to local build         
DEBU[0000] validating yamltags of struct SkaffoldConfig 
DEBU[0000] validating yamltags of struct Metadata       
DEBU[0000] validating yamltags of struct Pipeline       
DEBU[0000] validating yamltags of struct BuildConfig    
DEBU[0000] validating yamltags of struct Artifact       
DEBU[0000] validating yamltags of struct Sync           
DEBU[0000] validating yamltags of struct ArtifactType   
DEBU[0000] validating yamltags of struct DockerArtifact 
DEBU[0000] validating yamltags of struct Artifact       
DEBU[0000] validating yamltags of struct Sync           
DEBU[0000] validating yamltags of struct ArtifactType   
DEBU[0000] validating yamltags of struct DockerArtifact 
DEBU[0000] validating yamltags of struct Artifact       
DEBU[0000] validating yamltags of struct Sync           
DEBU[0000] validating yamltags of struct ArtifactType   
DEBU[0000] validating yamltags of struct DockerArtifact 
DEBU[0000] validating yamltags of struct Artifact       
DEBU[0000] validating yamltags of struct ArtifactType   
DEBU[0000] validating yamltags of struct DockerArtifact 
DEBU[0000] validating yamltags of struct Artifact       
DEBU[0000] validating yamltags of struct ArtifactType   
DEBU[0000] validating yamltags of struct DockerArtifact 
DEBU[0000] validating yamltags of struct TagPolicy      
DEBU[0000] validating yamltags of struct GitTagger      
DEBU[0000] validating yamltags of struct BuildType      
DEBU[0000] validating yamltags of struct LocalBuild     
DEBU[0000] validating yamltags of struct DeployConfig   
DEBU[0000] validating yamltags of struct DeployType     
DEBU[0000] validating yamltags of struct HelmDeploy     
DEBU[0000] validating yamltags of struct HelmRelease    
DEBU[0000] validating yamltags of struct HelmOverrides  
DEBU[0000] validating yamltags of struct HelmImageStrategy 
DEBU[0000] validating yamltags of struct HelmImageConfig 
DEBU[0000] validating yamltags of struct HelmDeployFlags 
INFO[0000] Using kubectl context: docker-desktop        
DEBU[0000] Using builder: local                         
DEBU[0000] setting Docker user agent to skaffold-v1.1.0 
DEBU[0000] push value not present, defaulting to false because localCluster is true 
Listing files to watch...
 - asseco/internet-banking-core
DEBU[0000] Found dependencies for dockerfile: [{package.json / true} {tsconfig.json / true} {package.json / true} {tsconfig.json / true} {. /app true} {. /app true}] 
 - asseco/internet-banking-accounts
DEBU[0000] Found dependencies for dockerfile: [{package.json / true} {tsconfig.json / true} {package.json / true} {tsconfig.json / true} {. /app true} {. /app true}] 
 - asseco/internet-banking-cards
DEBU[0000] Found dependencies for dockerfile: [{package.json / true} {tsconfig.json / true} {package.json / true} {tsconfig.json / true} {. /app true} {. /app true}] 
 - asseco/caop-proxy
DEBU[0000] Found dependencies for dockerfile: [{mock-server/package.json / true} {mock-server/yarn.lock / true} {mock-server/package.json / true} {mock-server/yarn.lock / true} {mock-server /app true} {caop-proxy-swagger /app/caop-proxy-swagger true} {mock-server /app true} {caop-proxy-swagger /app/caop-proxy-swagger true}] 
 - asseco/caop-proxy-graphql
DEBU[0000] Found dependencies for dockerfile: [{graphql/package.json / true} {graphql/yarn.lock / true} {graphql/package.json / true} {graphql/yarn.lock / true} {graphql /app true} {caop-proxy-swagger /app/caop-proxy-swagger true} {graphql /app true} {caop-proxy-swagger /app/caop-proxy-swagger true}] 
INFO[0000] List generated in 451.304473ms               
Generating tags...
 - asseco/internet-banking-core -> DEBU[0000] Running command: [git describe --tags --always] 
DEBU[0000] Running command: [git describe --tags --always] 
DEBU[0000] Running command: [git describe --tags --always] 
DEBU[0000] Running command: [git describe --tags --always] 
DEBU[0000] Running command: [git describe --tags --always] 
DEBU[0000] Command output: [ac70cf2
]                   
DEBU[0000] Running command: [git status . --porcelain]  
DEBU[0000] Command output: [ac70cf2
]                   
DEBU[0000] Running command: [git status . --porcelain]  
DEBU[0000] Command output: []                           
asseco/internet-banking-core:ac70cf2
 - asseco/internet-banking-accounts -> DEBU[0000] Command output: [ac70cf2
]                   
DEBU[0000] Running command: [git status . --porcelain]  
DEBU[0000] Command output: []                           
DEBU[0000] Command output: []                           
asseco/internet-banking-accounts:ac70cf2
 - asseco/internet-banking-cards -> asseco/internet-banking-cards:ac70cf2
 - asseco/caop-proxy -> DEBU[0001] Command output: [3.1.3-75-g0b4faa0ee
]       
DEBU[0001] Running command: [git status . --porcelain]  
DEBU[0001] Command output: [3.1.3-75-g0b4faa0ee
]       
DEBU[0001] Running command: [git status . --porcelain]  
DEBU[0001] Command output: []                           
DEBU[0001] Command output: []                           
asseco/caop-proxy:3.1.3-75-g0b4faa0ee
 - asseco/caop-proxy-graphql -> asseco/caop-proxy-graphql:3.1.3-75-g0b4faa0ee
INFO[0001] Tags generated in 311.025703ms               
Checking cache...
DEBU[0001] Found dependencies for dockerfile: [{mock-server/package.json / true} {mock-server/yarn.lock / true} {mock-server/package.json / true} {mock-server/yarn.lock / true} {mock-server /app true} {caop-proxy-swagger /app/caop-proxy-swagger true} {mock-server /app true} {caop-proxy-swagger /app/caop-proxy-swagger true}] 
DEBU[0001] Found dependencies for dockerfile: [{package.json / true} {tsconfig.json / true} {package.json / true} {tsconfig.json / true} {. /app true} {. /app true}] 
DEBU[0001] Found dependencies for dockerfile: [{graphql/package.json / true} {graphql/yarn.lock / true} {graphql/package.json / true} {graphql/yarn.lock / true} {graphql /app true} {caop-proxy-swagger /app/caop-proxy-swagger true} {graphql /app true} {caop-proxy-swagger /app/caop-proxy-swagger true}] 
DEBU[0001] Found dependencies for dockerfile: [{package.json / true} {tsconfig.json / true} {package.json / true} {tsconfig.json / true} {. /app true} {. /app true}] 
DEBU[0001] Found dependencies for dockerfile: [{package.json / true} {tsconfig.json / true} {package.json / true} {tsconfig.json / true} {. /app true} {. /app true}] 
 - asseco/internet-banking-core: Found Locally
 - asseco/internet-banking-accounts: Found Locally
 - asseco/internet-banking-cards: Found Locally
 - asseco/caop-proxy: Found Locally
 - asseco/caop-proxy-graphql: Found Locally
INFO[0002] Cache check complete in 1.573441504s         
Tags used in deployment:
 - asseco/internet-banking-core -> asseco/internet-banking-core:38942de56873e34c3370366bcfe939105f94e298a7be3ab171491e4f13e18e1a
 - asseco/internet-banking-accounts -> asseco/internet-banking-accounts:6803286f16ff235682c0edc090edbc163ca572a8184ea152fde5cfbf08b0104b
 - asseco/internet-banking-cards -> asseco/internet-banking-cards:897afb6c8815c75c12309c46d275d917e48ab24981eeaec2a633411c3900f956
 - asseco/caop-proxy -> asseco/caop-proxy:690a2524241b3f3fb001b2a514c4d9f62e293c8840aa45fe5fbf37cb177cb9b2
 - asseco/caop-proxy-graphql -> asseco/caop-proxy-graphql:e9dbabb805bc92cc40e88a8f9705867068cbba8f66e7bb67c3f184cfd00520eb
   local images can't be referenced by digest. They are tagged and referenced by a unique ID instead
Starting deploy...
DEBU[0002] Executing template &{envTemplate 0xc00027c100 0xc000096440  } with environment map[ANDROID_HOME:/Users/vitorcamacho/Library/Android/sdk COLORTERM:truecolor DISPLAY:/private/tmp/com.apple.launchd.wjEzSWqxo5/org.macosforge.xquartz:0 HOME:/Users/vitorcamacho LANG:en_GB.UTF-8 LC_CTYPE:en_GB.UTF-8 LESS:-R LOGNAME:vcamacho LSCOLORS:Gxfxcxdxbxegedabagacad NVM_CD_FLAGS:-q NVM_DIR:/Users/vitorcamacho/.nvm OLDPWD:/Users/vitorcamacho/workspace/use-cases/microfrontends PAGER:less PATH:/usr/local/opt/ruby/bin:/Users/vitorcamacho/.fastlane/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/opt/X11/bin:/usr/local/opt/ruby/bin:/Users/vitorcamacho/.fastlane/bin:/Users/vitorcamacho/Library/Android/sdk/tools:/Users/vitorcamacho/Library/Android/sdk/platform-tools:/Users/vitorcamacho/Library/Android/sdk/tools:/Users/vitorcamacho/Library/Android/sdk/platform-tools PWD:/Users/vitorcamacho/workspace/use-cases/microfrontends SHELL:/bin/zsh SHLVL:1 SSH_AUTH_SOCK:/private/tmp/com.apple.launchd.lbhGffGHwi/Listeners TERM:xterm-256color TERM_PROGRAM:vscode TERM_PROGRAM_VERSION:1.41.1 TMPDIR:/var/folders/s3/36cx9dvj5dqdd71v2gqv9j7m0000gp/T/ USER:vcamacho XPC_FLAGS:0x0 XPC_SERVICE_NAME:0 ZSH:/Users/vitorcamacho/.oh-my-zsh _:/usr/local/bin/skaffold __CF_USER_TEXT_ENCODING:0x1F6:0x0:0x0] 
DEBU[0003] Running command: [helm --kube-context docker-desktop get internet-banking] 
INFO[0003] Building helm dependencies...                
DEBU[0003] Running command: [helm --kube-context docker-desktop dep build .helm] 
DEBU[0003] EnvVarMap: map[string]string{"DIGEST":"asseco/internet-banking-core:38942de56873e34c3370366bcfe939105f94e298a7be3ab171491e4f13e18e1a", "DIGEST2":"asseco/internet-banking-accounts:6803286f16ff235682c0edc090edbc163ca572a8184ea152fde5cfbf08b0104b", "DIGEST3":"asseco/internet-banking-cards:897afb6c8815c75c12309c46d275d917e48ab24981eeaec2a633411c3900f956", "DIGEST4":"asseco/caop-proxy:690a2524241b3f3fb001b2a514c4d9f62e293c8840aa45fe5fbf37cb177cb9b2", "DIGEST5":"asseco/caop-proxy-graphql:e9dbabb805bc92cc40e88a8f9705867068cbba8f66e7bb67c3f184cfd00520eb", "DIGEST_ALGO":"asseco/internet-banking-core", "DIGEST_ALGO2":"asseco/internet-banking-accounts", "DIGEST_ALGO3":"asseco/internet-banking-cards", "DIGEST_ALGO4":"asseco/caop-proxy", "DIGEST_ALGO5":"asseco/caop-proxy-graphql", "DIGEST_HEX":"38942de56873e34c3370366bcfe939105f94e298a7be3ab171491e4f13e18e1a", "DIGEST_HEX2":"6803286f16ff235682c0edc090edbc163ca572a8184ea152fde5cfbf08b0104b", "DIGEST_HEX3":"897afb6c8815c75c12309c46d275d917e48ab24981eeaec2a633411c3900f956", "DIGEST_HEX4":"690a2524241b3f3fb001b2a514c4d9f62e293c8840aa45fe5fbf37cb177cb9b2", "DIGEST_HEX5":"e9dbabb805bc92cc40e88a8f9705867068cbba8f66e7bb67c3f184cfd00520eb", "IMAGE_NAME":"asseco/internet-banking-core", "IMAGE_NAME2":"asseco/internet-banking-accounts", "IMAGE_NAME3":"asseco/internet-banking-cards", "IMAGE_NAME4":"asseco/caop-proxy", "IMAGE_NAME5":"asseco/caop-proxy-graphql"} 
WARN[0003] image [asseco/internet-banking-core:38942de56873e34c3370366bcfe939105f94e298a7be3ab171491e4f13e18e1a] is not used. 
WARN[0003] image [asseco/internet-banking-core] is used instead. 
WARN[0003] See helm sample for how to replace image names with their actual tags: https://github.com/GoogleContainerTools/skaffold/blob/master/examples/helm-deployment/skaffold.yaml 
WARN[0003] image [asseco/internet-banking-accounts:6803286f16ff235682c0edc090edbc163ca572a8184ea152fde5cfbf08b0104b] is not used. 
WARN[0003] image [asseco/internet-banking-accounts] is used instead. 
WARN[0003] See helm sample for how to replace image names with their actual tags: https://github.com/GoogleContainerTools/skaffold/blob/master/examples/helm-deployment/skaffold.yaml 
WARN[0003] image [asseco/internet-banking-cards:897afb6c8815c75c12309c46d275d917e48ab24981eeaec2a633411c3900f956] is not used. 
WARN[0003] image [asseco/internet-banking-cards] is used instead. 
WARN[0003] See helm sample for how to replace image names with their actual tags: https://github.com/GoogleContainerTools/skaffold/blob/master/examples/helm-deployment/skaffold.yaml 
WARN[0003] image [asseco/caop-proxy:690a2524241b3f3fb001b2a514c4d9f62e293c8840aa45fe5fbf37cb177cb9b2] is not used. 
WARN[0003] image [asseco/caop-proxy] is used instead.   
WARN[0003] See helm sample for how to replace image names with their actual tags: https://github.com/GoogleContainerTools/skaffold/blob/master/examples/helm-deployment/skaffold.yaml 
WARN[0003] image [asseco/caop-proxy-graphql:e9dbabb805bc92cc40e88a8f9705867068cbba8f66e7bb67c3f184cfd00520eb] is not used. 
WARN[0004] image [asseco/caop-proxy-graphql] is used instead. 
WARN[0004] See helm sample for how to replace image names with their actual tags: https://github.com/GoogleContainerTools/skaffold/blob/master/examples/helm-deployment/skaffold.yaml 
DEBU[0004] Running command: [helm --kube-context docker-desktop upgrade internet-banking --force .helm] 
client.go:401: Replaced "caop-proxy-deployment" with kind Deployment for kind Deployment
client.go:401: Replaced "cards-deployment" with kind Deployment for kind Deployment
client.go:401: Replaced "accounts-deployment" with kind Deployment for kind Deployment
client.go:401: Replaced "caop-proxy-graphql-deployment" with kind Deployment for kind Deployment
client.go:401: Replaced "core-deployment" with kind Deployment for kind Deployment
Release "internet-banking" has been upgraded. Happy Helming!
NAME: internet-banking
LAST DEPLOYED: Thu Jan 16 09:59:03 2020
NAMESPACE: default
STATUS: deployed
REVISION: 53
TEST SUITE: None
DEBU[0006] Running command: [helm --kube-context docker-desktop get internet-banking] 
INFO[0006] error decoding parsed yaml: yaml: line 3: mapping values are not allowed in this context 
DEBU[0006] getting client config for kubeContext: ``    
DEBU[0006] getting client config for kubeContext: ``    
INFO[0006] Deploy complete in 4.030322058s              
DEBU[0006] getting client config for kubeContext: ``    
DEBU[0007] Change detected <nil>                        
DEBU[0007] Found dependencies for dockerfile: [{package.json / true} {tsconfig.json / true} {package.json / true} {tsconfig.json / true} {. /app true} {. /app true}] 
DEBU[0007] Found dependencies for dockerfile: [{package.json / true} {tsconfig.json / true} {package.json / true} {tsconfig.json / true} {. /app true} {. /app true}] 
DEBU[0007] Found dependencies for dockerfile: [{package.json / true} {tsconfig.json / true} {package.json / true} {tsconfig.json / true} {. /app true} {. /app true}] 
DEBU[0007] Found dependencies for dockerfile: [{mock-server/package.json / true} {mock-server/yarn.lock / true} {mock-server/package.json / true} {mock-server/yarn.lock / true} {mock-server /app true} {caop-proxy-swagger /app/caop-proxy-swagger true} {mock-server /app true} {caop-proxy-swagger /app/caop-proxy-swagger true}] 
DEBU[0007] Found dependencies for dockerfile: [{graphql/package.json / true} {graphql/yarn.lock / true} {graphql/package.json / true} {graphql/yarn.lock / true} {graphql /app true} {caop-proxy-swagger /app/caop-proxy-swagger true} {graphql /app true} {caop-proxy-swagger /app/caop-proxy-swagger true}]

And when I run kubectl get services,pods:

NAME                                            READY   STATUS             RESTARTS   AGE
accounts-deployment-64d4997fd5-dkg6r            0/1     ImagePullBackOff        
caop-proxy-deployment-56f95579b8-rkl5g          0/1     ImagePullBackOff   
caop-proxy-graphql-deployment-fb857b67b-4wm2p   0/1     ImagePullBackOff   
cards-deployment-7f45759948-hd7mf               0/1     ImagePullBackOff           
core-deployment-64b4646f4d-z8p8d                0/1     ImagePullBackOff
itechbear commented 4 years ago

Same issue here. skaffold run doesn't respect image locality or tag. It simply pulls image from dockerhub and uses the latest tag.

I googled a lot, all links stating that the root cause might be invalid .yaml files, but it turned out the local .yaml files are valid.

Finally I think I've got the root cause, the root cause might be skaffold's compatibility issue with helm v3, which I'm using.

For example, in the logs above, see this line:

DEBU[0006] Running command: [helm --kube-context docker-desktop get internet-banking] INFO[0006] error decoding parsed yaml: yaml: line 3: mapping values are not allowed in this context

The correct command should be

helm --kube-context docker-desktop get all internet-banking

With helm v3, you have to insert an all between get and the release name. If you miss it, the command would return an error message showing how to use the helm get command instead of a valid .yaml file.

itechbear commented 4 years ago

I switched to helm v2 + tiller, the result of the helm get command is valid now but we got a new problem.

time="2020-01-20T22:10:29+08:00" level=debug msg="Running command: [helm --kube-context minikube get cassandra]" time="2020-01-20T22:10:29+08:00" level=info msg="error decoding parsed yaml: Object 'Kind' is missing in 'REVISION: 2\nRELEASED: Mon Jan 20 22:10:29 2020\nCHART: cassandra-0.0.1\nUSER-SUPPLIED VALUES:\ncluster:\n domain: cluster.local\n\nCOMPUTED VALUES:\ncluster:\n domain: cluster.local\n\nHOOKS:\nMANIFEST:\n\n'" time="2020-01-20T22:10:29+08:00" level=warning msg="image [cassandra:fdd3c3dc0f7dbe3ab33611863fb778b0aff3669abfee15c3970cd2ec1951aaf6] is not used." time="2020-01-20T22:10:29+08:00" level=warning msg="image [cassandra] is used instead."

The first few lines of the returned .yaml format content should not parsed, but skaffold treats it as a kubernetes resource object and parsed it with erroneously.

FYI, the returned content:

REVISION: 2 RELEASED: Mon Jan 20 22:10:29 2020 CHART: cassandra-0.0.1 USER-SUPPLIED VALUES: cluster: domain: cluster.local

COMPUTED VALUES: cluster: domain: cluster.local

HOOKS: MANIFEST:

--- # Source: cassandra/templates/service.yaml apiVersion: v1 kind: Service ... and lots of k8s stuff

nkubala commented 4 years ago

hey @itechbear @vitorcamachoo and company, circling back on this one. one "quirk" with using helm with skaffold right now is that for skaffold to do image tag/default repo replacement on redeployment correctly, your image needs to be explicitly declared in your skaffold.yaml as a supplemental value that gets passed along to helm on the CLI:

deploy:
  helm:
    releases:
    - name: my_release
      values:
        image: my_image # this is required

I know this isn't a great UX since many devs configure their image repo/tag in the values.yaml, but it's the way helm was originally implemented in skaffold. I'm considering alternatives, but for now could y'all see if you could rework your skaffold project configuration and update if this addresses the issue?

itechbear commented 4 years ago

Hi, @nkubala , I confirm your solution works since skaffold v1.5.0, which comes with helm v3 support.

One thing to double check, with the above solution, is image: {{ .Values.image}} in pod definition still a must as of now?

mhn10 commented 4 years ago

I was facing the same issue and put in values.image in skaffold.yaml and its working now. But I have specified pullPolicy: Always in my Values.yaml for my remote clusters. Skaffold run fails with ErrImagePull. Would be great if skaffold can overwrite this pullPolicy from skaffold.yaml files like values.pullPolicy: Never

nkubala commented 4 years ago

@itechbear yep as of now that is required. I put together a short guide on using helm with skaffold which will get shipped into our docs in the next release: https://github.com/GoogleContainerTools/skaffold/pull/3973

@mhn10 you should be able to set the pullPolicy (as well as any arbitrary value) as a value in your skaffold.yaml as well - if this isn't working, it's probably a bug. I vaguely remember seeing another issue to track this...can't find it now, but if not feel free and open an issue.

I'm gonna close this issue for now as we're tracking helm improvements in other issues.

raul1991 commented 4 years ago

hey @itechbear @vitorcamachoo and company, circling back on this one. one "quirk" with using helm with skaffold right now is that for skaffold to do image tag/default repo replacement on redeployment correctly, your image needs to be explicitly declared in your skaffold.yaml as a supplemental value that gets passed along to helm on the CLI:

deploy:
  helm:
    releases:
    - name: my_release
      values:
        image: my_image # this is required

I know this isn't a great UX since many devs configure their image repo/tag in the values.yaml, but it's the way helm was originally implemented in skaffold. I'm considering alternatives, but for now could y'all see if you could rework your skaffold project configuration and update if this addresses the issue?

Just to clear my understanding, the values.image given in the skaffold yaml will replace the values in the values.yaml. So in my pod definition I should be using a {{ .Values.image }} to get the value overriden in the skaffold yaml file. Correct ?

and the --default-repo option works the way as it does with kubectl. So ultimately my image would be picked up from

/path/to/default-repo/{{.Values.image}} <- whatever this gets interpolated as. Correct ?

nkubala commented 4 years ago

@raul1991 at the time you were right, however we've renamed this field in the skaffold.yaml to artifactOverrides to make this more clear, and changes values to setValues:

deploy:
  helm:
    releases:
    - name: my_release
      artifactOverrides:
        image: my_image
      setValues:
        foo: bar