josmo / drone-rancher

Drone plugin for triggering Rancher deployments
53 stars 35 forks source link

The plugin isn't able to find my stacks anymore #32

Closed Vesli closed 6 years ago

Vesli commented 6 years ago

Hello, I've like 10 environments and each of them contains ~10 stacks (with 2 to 3 containers per stack) and everything was working well until I added a new Env.

In this rancher env I've added 8 stacks, when I deploy with drone, 7 of those stacks works well, but anything after this 7th return in drone:

Unable to find stack

I've added a 8th and 9th one.. none of them will be found by the plugin.

screen shot 2018-02-21 at 12 13 16
josmo commented 6 years ago

Hi @Vesli can you give details on the environment and the .drone.yml? I've never seen that problem before and I've had larger setups.

Vesli commented 6 years ago

Sure, let's take this example:

screen shot 2018-02-22 at 10 34 01

We have 5 stacks in this env. Each stacks has 1 to 3 containers - which I agree with you, is nothing terrific.

If I add one more stack to this env, drone result with my initial error "can't find the stack". Concerning the drone file, here is the one we recently used and had this error:

pipeline:

  ## Master
  push-master:
    image: plugins/docker
    secrets: [docker_username, docker_password]
    repo: docker.domain.com:5000/descartes
    registry: docker.domain.com:5000
    tags: "latest"
    when:
      event: push
      branch: master

  rancher-deploy-master:
    image: peloton/drone-rancher
    url: http://rancher.domain.com:8080
    secrets: [ plugin_access_key, plugin_secret_key ]
    service: descartes/nomad
    docker_image:  docker.heyolly.com:5000/descartes:latest
    start_first: true
    time_out: 120
    when:
      event: push
      branch: master

  ## Dev
  push-dev:
    image: plugins/docker
    secrets: [docker_username, docker_password]
    repo: docker.domain.com:5000/descartes
    registry: docker.domain.com:5000
    tags: "dev"
    when:
      event: push
      branch: dev

  rancher-deploy-dev:
    image: peloton/drone-rancher
    url: http://rancher.domain.com:8080
    secrets: [ plugin_access_key, plugin_secret_key ]
    service: descartes-dev/descartes
    docker_image:  docker.domain.com:5000/descartes:dev
    start_first: true
    time_out: 120
    when:
      event: push
      branch: dev

  ## Feature branches.
  push-feature:
    image: plugins/docker
    secrets: [docker_username, docker_password]
    repo: docker.domain.com:5000/descartes
    registry: docker.domain.com:5000
    tags: "feature"
    when:
      event: push
      branch:
        exclude: [ dev, master, test-* ]

    ## Testing branch
  push-staging:
    image: plugins/docker
    secrets: [docker_username, docker_password]
    repo: docker.domain.com:5000/descartes
    registry: docker.domain.com:5000
    tags: ${DRONE_COMMIT_BRANCH}
    when:
      branch: test-*

  rancher-deploy-staging:
    image: peloton/drone-rancher
    url: http://rancher.domain.com:8080
    secrets: [ plugin_access_key, plugin_secret_key ]
    service: vision-staging/Descartes #this was an other stack, but we had to remove it as it was producing the error***
    docker_image:  docker.domain.com:5000/descartes:${DRONE_COMMIT_BRANCH}
    start_first: true
    time_out: 120
    when:
      branch: test-*

  ###
   # NOTIFY build
  ###
  notify-build:
    image: plugins/slack
    webhook: https://hooks.slack.com/services/aosifu1948UIAO
    channel: jenkins
    username: The Mighty Pokeball
    template: >
      {{#success build.status}}
        Build {{build.link}} succeeded on {{repo.name}} from branch {{uppercase build.branch}}!
      {{else}}
        Build {{build.link}} failed! {{repo.name}} from branch {{uppercase build.branch}}
      {{/success}}
    when:
      status: [ failure ]

*** the stack name was Descartes-staging/descartes and was failing.

josmo commented 6 years ago

@Vesli Sorry, looking at it nothing jumps out :( . sadly I haven't heard or seen anything like this so without access to it, it would be pretty hard for me to trouble shoot :/ have you tried running the commands by hand (local docker container) and seeing if you see the same error? then you could use a proxy to see if anything obvious is there

josmo commented 6 years ago

@Vesli just wanted to check in if you ever found the issue?

Vesli commented 6 years ago

@josmo hey! thank you for your attention on it :) Actually I'm still not so sure and haven't tried something else, but I would assume it might come from an environment API that allows drone to upgrade on rancher. I need to setup more environment API keys and see how drone behave with those.

Vesli commented 6 years ago

Ok, so after spending some time on it I can confirm the problem: I was using a general key API from rancher, for all the env / stack.

I assume that there is a timeout or a parser problem somewhere that block the plugin from finding the stack.

But now, I'm using a specific key per env and it's working!

josmo commented 6 years ago

Cool Thanks for letting me know and closing it out :)