search

Shippable / support

Shippable SaaS customers can report issues and feature requests in this repository
100 stars 28 forks source link

Build failure with docker push using environment variable. #2521

Closed twillms07 closed 8 years ago

twillms07 commented 8 years ago

Is your request:

Unable to execute a docker push which I suspect is a result of the env var $BUILD_NUMBER

docker push https://977049052371.dkr.ecr.us-east-1.amazonaws.com/api:$BUILD_NUMBER  
Invalid repository name (ex: "registry.domain.tld/myrepos")

Link to build

# language setting
language: scala 

# version numbers, testing against two versions of node
scala:
    - 2.11.6
# Commented the line below to test the git_sync issues
# as recommended in Git Issue - https://github.com/Shippable/support/issues/2462
#build_image: drydock/u14sca:prod

build:
  #pre_ci:  
  #    - sbt clean
  #    - sbt assembly
  #    - cp server//target/scala-2.11/personas-server-assembly-1.0-SNAPSHOT.jar ./buildoutput/server.jar
  #    - cp server/src/main/resources/application-jdbc-psql.conf ./buildoutput/application.conf
  #    - docker build -t api:$BRANCH.$BUILD_NUMBER .

  post_ci_boot:
    #image_name: drydock/u14sca
    #image_tag: prod
    image_name: api
    image_tag: $$BUILD_NUMBER
    pull: false
    options: "-e HOME=/root"

  post_ci:
  #    - sbt clean
  #    - sbt assembly
  #    - cp server//target/scala-2.11/personas-server-assembly-1.0-SNAPSHOT.jar ./buildoutput/server.jar
  #    - cp server/src/main/resources/application-jdbc-psql.conf ./buildoutput/application.conf
      - docker build -t api:$BUILD_NUMBER .
      #- docker images
      - docker push https://977049052371.dkr.ecr.us-east-1.amazonaws.com/api:$BUILD_NUMBER
      - docker tag -f api:latest https://977049052371.dkr.ecr.us-east-1.amazonaws.com/api:$BUILD_NUMBER

  integrations:
    hub:
      - integrationName: amazon_ecr
        type: ecr
        agent_only: true

  ci:
    - sudo add-apt-repository ppa:webupd8team/java
    - sudo apt-get update
    - sudo apt-get install oracle-java8-installer
    - echo oracle-java8-installer shared/accepted-oracle-license-v1-1 select true | sudo /usr/bin/debconf-set-selections
    - sudo update-java-alternatives -s java-8-oracle
    - mkdir -p shippable/buildoutput
    #- sbt clean
    #- sbt assembly
    #- cp server//target/scala-2.11/personas-server-assembly-1.0-SNAPSHOT.jar ./shippable/buildoutput/server.jar
    #- cp server/src/main/resources/application-jdbc-psql.conf ./shippable/buildoutput/application.conf

  # script
    - export SBT_OPTS="-XX:+CMSClassUnloadingEnabled -XX:PermSize=256M -XX:MaxPermSize=512M"
    #- sbt clean test
    #- sbt clean coverage test coverageReport
    #- mkdir -p shippable/codecoverage
    #- mkdir -p shippable/test-reports
    #- mv target/scala-2.11/coverage-report/cobertura.xml $PWD/shippable/codecoverage/
    #- mv target/test-reports/* $PWD/shippable/test-reports/*
    #- mv target/scala-2.11/scoverage-report/* $PWD/shippable/test-reports
    - sbt clean
    #add various properties from shippable to make /info work correctly
    - echo "version=1.0.$BUILD_NUMBER" > server/src/main/resources/version.properties
    - echo "commit=$COMMIT" >> server/src/main/resources/version.properties
    - echo "branch=$BRANCH" >> server/src/main/resources/version.properties
    - echo "build_time=$LAST_SUCCESSFUL_BUILD_TIMESTAMP" >> server/src/main/resources/version.properties
    - cat server/src/main/resources/version.properties
    - sbt "test-only -l RequiresServer" assembly publish

  #on_success: 
  #after_script:
    - cp -r server/target/scala-2.11/personas-server-assembly-1.0-SNAPSHOT.jar ./shippable/buildoutput/server.jar
    - mkdir -p shippable/deployment
    #- cp server/src/main/resources/application-rds-dev.conf ./shippable/buildoutput/application.conf
    - cp server/src/main/resources/application-cassandra.conf ./shippable/buildoutput/application.conf
    - test -f ./shippable/buildoutput/server.jar

  jdk:
     - oraclejdk8

  cache: true

  archive: false

integrations:
    notifications:       
      - integrationName: email
        type: email
        recipients:
          - techadmin@piiq.co
        on_success: true
        on_failure: true
a-murphy commented 8 years ago

The variable $BUILD_NUMBER should have been filled in as expected when the command ran, but the command in the logs shows $BUILD_NUMBER because that's what's in the script.

It's much more likely the https:// that caused the error because docker push doesn't expect a protocol. Could you try docker push 977049052371.dkr.ecr.us-east-1.amazonaws.com/api:$BUILD_NUMBER?

twillms07 commented 8 years ago

Thanks we¹ll give it a try Dale-Kurt cc¹d here is out for the night and he¹s our lead DevOps person. We¹re trying to get our build out tonight so that our offshore team can begin integrating our app so I¹ll attempt to do my best.

a-murphy commented 8 years ago

Sorry, I didn't read your shippable.yml very closely. You'll need to switch the order of the docker push and the docker tag, too, because you're naming the image api:latest when you are building it. (You need to tag the image before pushing it with that name, and remove the https:// from that as well.)

So,docker tag -f api:latest 977049052371.dkr.ecr.us-east-1.amazonaws.com/api:$BUILD_NUMBER then docker push 977049052371.dkr.ecr.us-east-1.amazonaws.com/api:$BUILD_NUMBER.

twillms07 commented 8 years ago

We¹re still seeing build errors is there any insight you can provide? This all used to work! This is beginning to get incredibly frustrating and time consuming for our team. I¹ll forward the the issue we¹re seeing.

https://app.shippable.com/runs/570dd041b7923b0c00d92752

twillms07 commented 8 years ago

Todd Willms | Founder & CEO | PiiQ twillms@piiq.co mailto:twillms@piiq.co | Mobile: 415 871 5171

From: noreply@shippable.com on behalf of "Shippable.com" noreply@shippable.com Date: Tuesday, April 12, 2016 at 10:09 PM To: techadmin@piiq.co Subject: [ develop ] Build FAILED for PiiQ-Personas/personas-server ( Update shippable.yml ) on Shippable.com

 <https://www.shippable.com>

BUILD REPORT
Repo personas-server Repo URL git@github.com:PiiQ-Personas/personas-server.git Language scala Triggered By Todd Committed By Todd Author Todd
Commit Message Update shippable.yml Branch develop Changes 76858ab44fd22e32cee294c68b7188a022a2f87a https://github.com/PiiQ-Personas/personas-server/commit/76858ab44fd22e32cee 294c68b7188a022a2f87a Build triggered from a commit

Runtime Status Tests Branch Coverage Sequence Coverage

Triggered at Duration Details 436.1 2.11.6 failed 0/0 N/A N/A Wed Apr 13 2016 04:51:13 GMT+0000 (UTC) 17.94 minutes ..more https://app.shippable.com/runs/570dd041b7923b0c00d92752

Regards, Team Shippable

<https://www.facebook.com/beShippable>

http://www.linkedin.com/company/shippable https://twitter.com/beShippable

Need help with Shippable? File an Issue https://github.com/Shippable/support/issues

a-murphy commented 8 years ago

It looks like you're trying to push an image that doesn't exist. Docker doesn't have an image called 977049052371.dkr.ecr.us-east-1.amazonaws.com/api:$BUILD_NUMBER locally, so it can't figure out what you want pushed. You could either change the name (after -t) in docker build to match what you are trying to push or tag (docker tag api:$BUILD_NUMBER 977049052371.dkr.ecr.us-east-1.amazonaws.com/api:$BUILD_NUMBER) the image before pushing it so that it will have the right name. I'll have to look at how it was working for you before. Were you using a different container registry?

twillms07 commented 8 years ago

No we were just tagging it with latest not the build number.

From: a-murphy notifications@github.com Reply-To: Shippable/support <reply+004a65c5250bb6e3e88c7a234140e7789140c857671ea82d92cf000000011325997c9 2a169ce08d12dc9@reply.github.com> Date: Tuesday, April 12, 2016 at 10:22 PM To: Shippable/support support@noreply.github.com Cc: Todd Willms twillms07@gmail.com Subject: Re: [Shippable/support] Build failure with docker push using environment variable. (#2521)

It looks like you're trying to push an image that doesn't exist. Docker doesn't have an image called 977049052371.dkr.ecr.us-east-1.amazonaws.com/api:$BUILD_NUMBER locally, so it can't figure out what you want pushed. You could either change the name (after -t) in docker build to match what you are trying to push or tag (docker tag api:$BUILD_NUMBER 977049052371.dkr.ecr.us-east-1.amazonaws.com/api:$BUILD_NUMBER) the image before pushing it so that it will have the right name. I'll have to look at how it was working for you before. Were you using a different container registry?

‹ You are receiving this because you authored the thread. Reply to this email directly or view it on GitHub https://github.com/Shippable/support/issues/2521#issuecomment-209232020

manishas commented 8 years ago

@twillms07 we need to look into several issues here.

[1] CI build image Which image do you want to use for CI? Currently, we're pulling our standard Scala build image drydock/u14sca:prod for your build.

[2] What image do you want to push to your ECR registry?

You need to build the api image with the fully qualified name and push it like this:

build:
  post_ci:
    - docker build -t 977049052371.dkr.ecr.us-east-1.amazonaws.com/api:$BUILD_NUMBER .  
    - docker push 977049052371.dkr.ecr.us-east-1.amazonaws.com/api:$BUILD_NUMBER

Overall, I took a stab at your yml and here is what I think it should be. If this doesn't do what you need, we'll be happy to jump on a hangout and help debug real time. Please note that I have commented out the agent_only: true since you're trying to push to ECR from the post_ci section.

# language setting
language: scala 

# version numbers, testing against two versions of node
scala:
    - 2.11.6

jdk:
   - oraclejdk8

build:
  ci:
    - sudo add-apt-repository ppa:webupd8team/java
    - sudo apt-get update
    - sudo apt-get install oracle-java8-installer
    - echo oracle-java8-installer shared/accepted-oracle-license-v1-1 select true | sudo /usr/bin/debconf-set-selections
    - sudo update-java-alternatives -s java-8-oracle
    - mkdir -p shippable/buildoutput
    - export SBT_OPTS="-XX:+CMSClassUnloadingEnabled -XX:PermSize=256M -XX:MaxPermSize=512M"
    - sbt clean
    #add various properties from shippable to make /info work correctly
    - echo "version=1.0.$BUILD_NUMBER" > server/src/main/resources/version.properties
    - echo "commit=$COMMIT" >> server/src/main/resources/version.properties
    - echo "branch=$BRANCH" >> server/src/main/resources/version.properties
    - echo "build_time=$LAST_SUCCESSFUL_BUILD_TIMESTAMP" >> server/src/main/resources/version.properties
    - cat server/src/main/resources/version.properties
    - sbt "test-only -l RequiresServer" assembly publish
    - cp -r server/target/scala-2.11/personas-server-assembly-1.0-SNAPSHOT.jar ./shippable/buildoutput/server.jar
    - mkdir -p shippable/deployment
    #- cp server/src/main/resources/application-rds-dev.conf ./shippable/buildoutput/application.conf
    - cp server/src/main/resources/application-cassandra.conf ./shippable/buildoutput/application.conf
    - test -f ./shippable/buildoutput/server.jar

  post_ci:
      - docker build -t api:$BUILD_NUMBER .
      - docker tag -f api:$BUILD_NUMBER 977049052371.dkr.ecr.us-east-1.amazonaws.com/api:$BUILD_NUMBER
      - docker push 977049052371.dkr.ecr.us-east-1.amazonaws.com/api:$BUILD_NUMBER

  cache: true

integrations:
    hub:
      - integrationName: amazon_ecr
        type: ecr
        #agent_only: true
    notifications:       
      - integrationName: email
        type: email
        recipients:
          - techadmin@piiq.co
        on_success: true
        on_failure: true
twillms07 commented 8 years ago

Thanks guys. We¹ll look at it this morning. Our team ran out of steam last night.

From: Manisha notifications@github.com Reply-To: Shippable/support <reply+004a65c5fa471ceffb5f235eef06e387915ce7b67b2e86e592cf000000011325a1d09 2a169ce08d12dc9@reply.github.com> Date: Tuesday, April 12, 2016 at 10:57 PM To: Shippable/support support@noreply.github.com Cc: Todd Willms twillms07@gmail.com Subject: Re: [Shippable/support] Build failure with docker push using environment variable. (#2521)

@twillms07 https://github.com/twillms07 we need to look into several issues here.

[1] CI build image Which image do you want to use for CI? Currently, we're pulling our standard Scala build image drydock/u14sca:prod for your build.

[2] What image do you want to push to your ECR registry?

You need to build the api image with the fully qualified name and push it like this: build: post_ci: docker build -t 977049052371.dkr.ecr.us-east-1.amazonaws.com/api:$BUILD_NUMBER . docker push 977049052371.dkr.ecr.us-east-1.amazonaws.com/api:$BUILD_NUMBER

Overall, I took a stab at your yml and here is what I think it should be. If this doesn't do what you need, we'll be happy to jump on a hangout and help debug real time.

language setting

language: scala

version numbers, testing against two versions of node

scala:

jdk:

build: ci:

integrations: hub:

‹ You are receiving this because you were mentioned. Reply to this email directly or view it on GitHub https://github.com/Shippable/support/issues/2521#issuecomment-209240528

twillms07 commented 8 years ago

Hi guys -

We received another failureŠ It would be great if we could jump on a hangout and just figure this out. We need to get this figured out before we do our alpha launch so that it runs seamless. I¹m available all morning PST to go through this.

Thanks

Todd Willms | Founder & CEO | PiiQ twillms@piiq.co mailto:twillms@piiq.co | Mobile: 415 871 5171

From: Todd Willms twillms07@gmail.com Date: Wednesday, April 13, 2016 at 7:03 AM To: Shippable/support <reply+004a65c5fa471ceffb5f235eef06e387915ce7b67b2e86e592cf000000011325a1d09 2a169ce08d12dc9@reply.github.com>, Shippable/support support@noreply.github.com Cc: Dale-Kurt Murray hello@dalekurtmurray.com, Todd Willms twillms@piiq.co Subject: Re: [Shippable/support] Build failure with docker push using environment variable. (#2521)

Thanks guys. We¹ll look at it this morning. Our team ran out of steam last night.

From: Manisha notifications@github.com Reply-To: Shippable/support <reply+004a65c5fa471ceffb5f235eef06e387915ce7b67b2e86e592cf000000011325a1d09 2a169ce08d12dc9@reply.github.com> Date: Tuesday, April 12, 2016 at 10:57 PM To: Shippable/support support@noreply.github.com Cc: Todd Willms twillms07@gmail.com Subject: Re: [Shippable/support] Build failure with docker push using environment variable. (#2521)

@twillms07 https://github.com/twillms07 we need to look into several issues here.

[1] CI build image Which image do you want to use for CI? Currently, we're pulling our standard Scala build image drydock/u14sca:prod for your build.

[2] What image do you want to push to your ECR registry?

You need to build the api image with the fully qualified name and push it like this: build: post_ci: docker build -t 977049052371.dkr.ecr.us-east-1.amazonaws.com/api:$BUILD_NUMBER . docker push 977049052371.dkr.ecr.us-east-1.amazonaws.com/api:$BUILD_NUMBER

Overall, I took a stab at your yml and here is what I think it should be. If this doesn't do what you need, we'll be happy to jump on a hangout and help debug real time.

language setting

language: scala

version numbers, testing against two versions of node

scala:

jdk:

build: ci:

integrations: hub:

‹ You are receiving this because you were mentioned. Reply to this email directly or view it on GitHub https://github.com/Shippable/support/issues/2521#issuecomment-209240528

manishas commented 8 years ago

Can you do 10am PST?

I did make one correction to the yml a few minutes after I posted it and I'm wondering if that's the culprit. Can you confirm that you've commented out the agent_only: true in the integrations section?

manishas commented 8 years ago

@twillms07 We have set up a hangout to help resolve this at 10am PST. Link is https://plus.google.com/hangouts/_/shippable.com/customersuccess?authuser=0

twillms07 commented 8 years ago

Awesome thank you I'll be there. On Apr 13, 2016 9:44 AM, "Manisha" notifications@github.com wrote:

@twillms07 https://github.com/twillms07 We have set up a hangout to help resolve this at 10am PST. Link is https://plus.google.com/hangouts/_/shippable.com/customersuccess?authuser=0

— You are receiving this because you were mentioned. Reply to this email directly or view it on GitHub https://github.com/Shippable/support/issues/2521#issuecomment-209537896

manishas commented 8 years ago

This has been resolved after removing the agent_only: true from the yml.

Closing this issue.