search

travis-ci / travis-ci

Free continuous integration platform for GitHub projects.
https://travis-ci.org
8.42k stars 719 forks source link

SBT launcher download fails #4527

Closed plamola closed 9 years ago

plamola commented 9 years ago

SBT builds fail on download of sbt-launcher:

$ sbt ++$TRAVIS_SCALA_VERSION -J-XX:ReservedCodeCacheSize=256M clean coverage test
Detected sbt version 0.13.8
Downloading sbt launcher for 0.13.8:
  From  http://typesafe.artifactoryonline.com/typesafe/ivy-releases/org.scala-sbt/sbt-launch/0.13.8/sbt-launch.jar
    To  /home/travis/.sbt/launchers/0.13.8/sbt-launch.jar
Download failed. Obtain the jar manually and place it at /home/travis/.sbt/launchers/0.13.8/sbt-launch.jar

Full build output at: https://travis-ci.org/magneticio/vamp-common/builds/72070114

I think the reason can be found here: https://github.com/travis-ci/travis-cookbooks/pull/491

plamola commented 9 years ago

Work around is downloading the correct sbt-launch.jar via a before_scriptentry.

For 0.13.8 this would be:

before_script:
- curl -L -o $HOME/.sbt/launchers/0.13.8/sbt-launch.jar http://dl.bintray.com/typesafe/ivy-releases/org.scala-sbt/sbt-launch/0.13.8/sbt-launch.jar
BanzaiMan commented 9 years ago

@plamola I'm unfamiliar with Scala ecosystem, so I would like to understand what is causing your build to pull the JAR from typesafe.artifactoryonline.com. That URL returns 404. Perhaps it is an incomplete mirror?

plamola commented 9 years ago

Typesafe announced in April the'll be moving their repo to bintray.

https://www.typesafe.com/blog/migrating-repos-to-bintray

Travis CI is using the old repo. As of today, this repo appears to be offline.

The travis cookbook for sbt-extra has been updated to fix this (see: travis-ci/travis-cookbooks#491), but maybe hasn't been rolled out to the VM's?

dwijnand commented 9 years ago

@plamola Correct. @eed3si9n reached out to Travis on Twitter, but they indicated no current ETA.

BanzaiMan commented 9 years ago

Ah, I see. Thank you for the information. You are correct to point out that we have not rolled out build environment update to reflect the change.

I'll look into fixing this in the meantime. Thanks.

dwijnand commented 9 years ago

Ah cool, thanks @BanzaiMan

andreionut commented 9 years ago

Any ETA on this as the builds still fail?

darkfrog26 commented 9 years ago

@andreionut I'm using the work-around prescribed above and that's doing the trick for me.

andreionut commented 9 years ago

That's fine for now, and I am using the work-around, but I'm waiting on a proper fix from Travis

rdenmark commented 9 years ago

Thank you, @plamola, for referencing the bintray location. Add one more to those inquiring on an ETA for a proper fix.

retronym commented 9 years ago

@rdenmark I would recommend to use the repo.typesafe.com URL as shown below. This is the canonical way to reference the repository, which currently happens to redirect to bintray.

curl -L -o $HOME/.sbt/launchers/0.13.8/sbt-launch.jar http://repo.typesafe.com/typesafe/ivy-releases/org.scala-sbt/sbt-launch/0.13.8/sbt-launch.jar
eed3si9n commented 9 years ago

.@retronym is right. Using the redirection target of repo.typesafe.com directly got us here in the first place, so we should avoid repeating that in the workaround.

Here's the full example with https and mkdir, which was needed for me.

before_script:
  - mkdir -p $HOME/.sbt/launchers/0.13.8/
  - curl -L -o $HOME/.sbt/launchers/0.13.8/sbt-launch.jar https://repo.typesafe.com/typesafe/ivy-releases/org.scala-sbt/sbt-launch/0.13.8/sbt-launch.jar
BardurArantsson commented 9 years ago

All, just FYI: curl has an option --create-dirs which will do the right thing so that you don't have do a separate mkdir -p.

BanzaiMan commented 9 years ago

Hello there. The fix has been deploy to production (both .org and .com).

Please try your builds again (even the previously failing ones). Let us know if you have problems.

Thank you!

eed3si9n commented 9 years ago

@BanzaiMan Thanks for the fix! It seems to be working for me.

pjan commented 9 years ago

@BanzaiMan builds still failing here, but now because of the non-resolution of sbt plugins

[warn]  ::::::::::::::::::::::::::::::::::::::::::::::
[warn]  ::          UNRESOLVED DEPENDENCIES         ::
[warn]  ::::::::::::::::::::::::::::::::::::::::::::::
[warn]  :: com.eed3si9n#sbt-unidoc;0.3.2: not found
[warn]  :: com.eed3si9n#sbt-assembly;0.13.0: not found
[warn]  :: com.github.gseitz#sbt-release;1.0.0: not found
[warn]  :: com.typesafe.sbt#sbt-native-packager;1.0.1: not found
[warn]  :: com.jsuereth#sbt-pgp;1.0.0: not found
[warn]  :: org.scoverage#sbt-scoverage;1.0.4: not found
[warn]  :: com.typesafe.sbt#sbt-scalariform;1.3.0: not found
[warn]  :: com.timushev.sbt#sbt-updates;0.1.8: not found
[warn]  ::::::::::::::::::::::::::::::::::::::::::::::
[warn] 
[warn]  Note: Some unresolved dependencies have extra attributes.  Check that these dependencies exist with the requested attributes.
[warn]      com.timushev.sbt:sbt-updates:0.1.8 (scalaVersion=2.10, sbtVersion=0.13)
[warn]      com.eed3si9n:sbt-assembly:0.13.0 (scalaVersion=2.10, sbtVersion=0.13)
[warn]      com.eed3si9n:sbt-unidoc:0.3.2 (scalaVersion=2.10, sbtVersion=0.13)
[warn]      com.jsuereth:sbt-pgp:1.0.0 (scalaVersion=2.10, sbtVersion=0.13)
[warn]      com.typesafe.sbt:sbt-scalariform:1.3.0 (scalaVersion=2.10, sbtVersion=0.13)
[warn]      org.scoverage:sbt-scoverage:1.0.4 (scalaVersion=2.10, sbtVersion=0.13)
[warn]      com.github.gseitz:sbt-release:1.0.0 (scalaVersion=2.10, sbtVersion=0.13)
[warn]      com.typesafe.sbt:sbt-native-packager:1.0.1 (scalaVersion=2.10, sbtVersion=0.13)
[warn] 
[warn]  Note: Unresolved dependencies path:
[warn]      com.timushev.sbt:sbt-updates:0.1.8 (scalaVersion=2.10, sbtVersion=0.13) (/home/travis/build/ExchangeCorporation/paidy-common-new/project/plugins.sbt#L20-21)
[warn]        +- default:paidy-common-new-build:0.1-SNAPSHOT (scalaVersion=2.10, sbtVersion=0.13)
[warn]      com.eed3si9n:sbt-assembly:0.13.0 (scalaVersion=2.10, sbtVersion=0.13) (/home/travis/build/ExchangeCorporation/paidy-common-new/project/plugins.sbt#L5-6)
[warn]        +- default:paidy-common-new-build:0.1-SNAPSHOT (scalaVersion=2.10, sbtVersion=0.13)
[warn]      com.eed3si9n:sbt-unidoc:0.3.2 (scalaVersion=2.10, sbtVersion=0.13) (/home/travis/build/ExchangeCorporation/paidy-common-new/project/plugins.sbt#L4-5)
[warn]        +- default:paidy-common-new-build:0.1-SNAPSHOT (scalaVersion=2.10, sbtVersion=0.13)
[warn]      com.jsuereth:sbt-pgp:1.0.0 (scalaVersion=2.10, sbtVersion=0.13) (/home/travis/build/ExchangeCorporation/paidy-common-new/project/plugins.sbt#L11-12)
[warn]        +- default:paidy-common-new-build:0.1-SNAPSHOT (scalaVersion=2.10, sbtVersion=0.13)
[warn]      com.typesafe.sbt:sbt-scalariform:1.3.0 (scalaVersion=2.10, sbtVersion=0.13) (/home/travis/build/ExchangeCorporation/paidy-common-new/project/plugins.sbt#L18-19)
[warn]        +- default:paidy-common-new-build:0.1-SNAPSHOT (scalaVersion=2.10, sbtVersion=0.13)
[warn]      org.scoverage:sbt-scoverage:1.0.4 (scalaVersion=2.10, sbtVersion=0.13) (/home/travis/build/ExchangeCorporation/paidy-common-new/project/plugins.sbt#L14-15)
[warn]        +- default:paidy-common-new-build:0.1-SNAPSHOT (scalaVersion=2.10, sbtVersion=0.13)
[warn]      com.github.gseitz:sbt-release:1.0.0 (scalaVersion=2.10, sbtVersion=0.13) (/home/travis/build/ExchangeCorporation/paidy-common-new/project/plugins.sbt#L6-7)
[warn]        +- default:paidy-common-new-build:0.1-SNAPSHOT (scalaVersion=2.10, sbtVersion=0.13)
[warn]      com.typesafe.sbt:sbt-native-packager:1.0.1 (scalaVersion=2.10, sbtVersion=0.13) (/home/travis/build/ExchangeCorporation/paidy-common-new/project/plugins.sbt#L7-8)
[warn]        +- default:paidy-common-new-build:0.1-SNAPSHOT (scalaVersion=2.10, sbtVersion=0.13)
sbt.ResolveException: unresolved dependency: com.eed3si9n#sbt-unidoc;0.3.2: not found
unresolved dependency: com.eed3si9n#sbt-assembly;0.13.0: not found
unresolved dependency: com.github.gseitz#sbt-release;1.0.0: not found
unresolved dependency: com.typesafe.sbt#sbt-native-packager;1.0.1: not found
unresolved dependency: com.jsuereth#sbt-pgp;1.0.0: not found
unresolved dependency: org.scoverage#sbt-scoverage;1.0.4: not found
unresolved dependency: com.typesafe.sbt#sbt-scalariform;1.3.0: not found
unresolved dependency: com.timushev.sbt#sbt-updates;0.1.8: not found
    at sbt.IvyActions$.sbt$IvyActions$$resolve(IvyActions.scala:291)
    at sbt.IvyActions$$anonfun$updateEither$1.apply(IvyActions.scala:188)
    at sbt.IvyActions$$anonfun$updateEither$1.apply(IvyActions.scala:165)
    at sbt.IvySbt$Module$$anonfun$withModule$1.apply(Ivy.scala:155)
    at sbt.IvySbt$Module$$anonfun$withModule$1.apply(Ivy.scala:155)
    at sbt.IvySbt$$anonfun$withIvy$1.apply(Ivy.scala:132)
    at sbt.IvySbt.sbt$IvySbt$$action$1(Ivy.scala:57)
    at sbt.IvySbt$$anon$4.call(Ivy.scala:65)
    at xsbt.boot.Locks$GlobalLock.withChannel$1(Locks.scala:93)
    at xsbt.boot.Locks$GlobalLock.xsbt$boot$Locks$GlobalLock$$withChannelRetries$1(Locks.scala:78)
    at xsbt.boot.Locks$GlobalLock$$anonfun$withFileLock$1.apply(Locks.scala:97)
    at xsbt.boot.Using$.withResource(Using.scala:10)
    at xsbt.boot.Using$.apply(Using.scala:9)
    at xsbt.boot.Locks$GlobalLock.ignoringDeadlockAvoided(Locks.scala:58)
    at xsbt.boot.Locks$GlobalLock.withLock(Locks.scala:48)
    at xsbt.boot.Locks$.apply0(Locks.scala:31)
    at xsbt.boot.Locks$.apply(Locks.scala:28)
    at sbt.IvySbt.withDefaultLogger(Ivy.scala:65)
    at sbt.IvySbt.withIvy(Ivy.scala:127)
    at sbt.IvySbt.withIvy(Ivy.scala:124)
    at sbt.IvySbt$Module.withModule(Ivy.scala:155)
    at sbt.IvyActions$.updateEither(IvyActions.scala:165)
    at sbt.Classpaths$$anonfun$sbt$Classpaths$$work$1$1.apply(Defaults.scala:1369)
    at sbt.Classpaths$$anonfun$sbt$Classpaths$$work$1$1.apply(Defaults.scala:1365)
    at sbt.Classpaths$$anonfun$doWork$1$1$$anonfun$87.apply(Defaults.scala:1399)
    at sbt.Classpaths$$anonfun$doWork$1$1$$anonfun$87.apply(Defaults.scala:1397)
    at sbt.Tracked$$anonfun$lastOutput$1.apply(Tracked.scala:37)
    at sbt.Classpaths$$anonfun$doWork$1$1.apply(Defaults.scala:1402)
    at sbt.Classpaths$$anonfun$doWork$1$1.apply(Defaults.scala:1396)
    at sbt.Tracked$$anonfun$inputChanged$1.apply(Tracked.scala:60)
    at sbt.Classpaths$.cachedUpdate(Defaults.scala:1419)
    at sbt.Classpaths$$anonfun$updateTask$1.apply(Defaults.scala:1348)
    at sbt.Classpaths$$anonfun$updateTask$1.apply(Defaults.scala:1310)
    at scala.Function1$$anonfun$compose$1.apply(Function1.scala:47)
    at sbt.$tilde$greater$$anonfun$$u2219$1.apply(TypeFunctions.scala:40)
    at sbt.std.Transform$$anon$4.work(System.scala:63)
    at sbt.Execute$$anonfun$submit$1$$anonfun$apply$1.apply(Execute.scala:226)
    at sbt.Execute$$anonfun$submit$1$$anonfun$apply$1.apply(Execute.scala:226)
    at sbt.ErrorHandling$.wideConvert(ErrorHandling.scala:17)
    at sbt.Execute.work(Execute.scala:235)
    at sbt.Execute$$anonfun$submit$1.apply(Execute.scala:226)
    at sbt.Execute$$anonfun$submit$1.apply(Execute.scala:226)
    at sbt.ConcurrentRestrictions$$anon$4$$anonfun$1.apply(ConcurrentRestrictions.scala:159)
    at sbt.CompletionService$$anon$2.call(CompletionService.scala:28)
    at java.util.concurrent.FutureTask.run(FutureTask.java:266)
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
    at java.util.concurrent.FutureTask.run(FutureTask.java:266)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
    at java.lang.Thread.run(Thread.java:745)
[error] (*:update) sbt.ResolveException: unresolved dependency: com.eed3si9n#sbt-unidoc;0.3.2: not found
[error] unresolved dependency: com.eed3si9n#sbt-assembly;0.13.0: not found
[error] unresolved dependency: com.github.gseitz#sbt-release;1.0.0: not found
[error] unresolved dependency: com.typesafe.sbt#sbt-native-packager;1.0.1: not found
[error] unresolved dependency: com.jsuereth#sbt-pgp;1.0.0: not found
[error] unresolved dependency: org.scoverage#sbt-scoverage;1.0.4: not found
[error] unresolved dependency: com.typesafe.sbt#sbt-scalariform;1.3.0: not found
[error] unresolved dependency: com.timushev.sbt#sbt-updates;0.1.8: not found
BanzaiMan commented 9 years ago

@pjan Is this something we should fix? I am not sure where the resolution failure is coming from.

rrdelaney commented 9 years ago

I'm getting the same thing on https://travis-ci.org/rrdelaney/vegas/builds/72308679

It looks like plugins are trying to be resolved with Scala 2.10

davidrpugh commented 9 years ago

Same as well on https://travis-ci.org/ScalABM/commons-sandbox.

pjan commented 9 years ago

@BanzaiMan it used to work before. Haven't changed anything in our build configuration.

BanzaiMan commented 9 years ago

@rrdelaney Hmmm. $TRAVIS_SCALA_VERSION should be set to 2.11.6, so that sbt should be invoked as sbt ++2.11.6 clean coverage test. Is this a problem with https://raw.githubusercontent.com/paulp/sbt-extras/b9c8cb273d38e0d8da9211902a18018fe82aa14e/sbt?

At any rate, it seems like a different problem to me. Could you open a separate ticket, please?

rtyley commented 9 years ago

I think it's actually normal for current sbt plugins to be resolved against Scala 2.10 (because that's what SBT 0.13 actually runs against)

...but, it looks like the most recent artifacts (Play, etc) have only been published to the old artifactory repository, not the new bintray one?

Old stuff is there: https://dl.bintray.com/typesafe/ivy-releases/com.typesafe.play/sbt-plugin/scala_2.10/sbt_0.13/2.3.8/

New stuff is not there: https://dl.bintray.com/typesafe/ivy-releases/com.typesafe.play/sbt-plugin/scala_2.10/sbt_0.13/2.4.2/

This is not travis's fault - more down to Typesafe I think.

ericluria commented 9 years ago

@pjan @rtyley @rrdelaney I ran into a similar issue and had to add the following resolver to plugins.sbt:

resolvers += Resolver.url("fix-sbt-plugin-releases", url("https://dl.bintray.com/sbt/sbt-plugin-releases"))(Resolver.ivyStylePatterns)

edit: use https instead of http

eed3si9n commented 9 years ago

@pjan @BanzaiMan sbt plugin resolution issues is unrelated problem that's our fault. sbt/sbt#2125 We will try to resolve this soon.

davidrpugh commented 9 years ago

@ericluria

Your temp solution worked for me...

edmundoa commented 9 years ago

Hi,

We are still experiencing the original issue, as you can see in this build: https://travis-ci.org/Graylog2/graylog2-web-interface/jobs/72444844. Do we need to take any steps on our side to fix the problem?

Thank you!

BanzaiMan commented 9 years ago

@edmundoa The fix does not update sbt on non-Scala builds. Is using language: scala an option for you? (This will cause cache to be refreshed.)

For non-Scala builds, sbt is far unlikely to be used, so we do not waste extra network calls. If you can't use language: scala, you can simply download the desired script and set it up yourself: https://github.com/travis-ci/travis-build/blob/bf2c5d76e95a6a4f895c71e7f6051a1df93ddeee/lib/travis/build/script/scala.rb#L21 (fetch the file, make it executable, and add it to your $PATH) until the image is updated.

BanzaiMan commented 9 years ago

@edmundoa Or an earlier workaround (https://github.com/travis-ci/travis-ci/issues/4527#issuecomment-123641768) may work, too.

edmundoa commented 9 years ago

@BanzaiMan I think the easiest way is to download the sbt launcher before the build. Thank you for your help! :+1:

muuki88 commented 7 years ago

Hi. I'm running in this issue once again. Everything works for me with the default configuration, but adding sudo: required triggers the mentioned bug.

The configuration in the include matrix looks like this

   - jdk: oraclejdk8
       services: docker
       sudo: required
       env: SBT_TEST_CMD="^validateDocker"

See: https://travis-ci.org/sbt/sbt-native-packager/jobs/295077801

raboof commented 5 years ago

I got this error when trying to use sbt 1.2.8 when I forgot to specify language: scala

t3hnar commented 5 years ago

@raboof I think this is a random issue and has nothing todo with language: scala, for instance I have this line in .yml and still can sometimes see the failure

dwijnand commented 5 years ago

It would seem that maybe there are multiple versions of sbt-extra scripts on different base images. The other issue where these kinds of problems are being discussed is #9816, where using language: scala is recommended.

raboof commented 5 years ago

Interesting. The issue I'm having today appears to simply be dl.bintray.com returning 502 errors (which is probably also the intermittent failure @t3hnar is seeing), but previously I definitely had consistent failures without language: scala which got fixed after adding it.

I guess the error message doesn't reveals whether the error was a 4xx or a 5xx, making it a bit ambiguous.

t3hnar commented 5 years ago

Today issue: https://status.bintray.com

http://prntscr.com/n0w9zw

channingwalton commented 5 years ago

Definitely intermittent, I tried a build a few times and it eventually worked.

dwijnand commented 5 years ago

You should make sure to cache the sbt launcher. The recommended configuration is:

cache:
  directories:
    - $HOME/.ivy2/cache
    - $HOME/.sbt

With that the only time this is an issue is when upgrading sbt version (where sbt-extras will download a new launcher jar into ~/.sbt/launchers), or starting a new project.

t3hnar commented 5 years ago

With that the only time this is an issue is when upgrading sbt version

This is what I did today 😂

scalahub commented 3 years ago

I'm having the same issue today for a new project https://travis-ci.com/github/scalahub/AppkitIssue/jobs/499438499#L209

denzhel commented 3 years ago

Same. Please fix it.

janjaali commented 3 years ago

Maybe as a workaround one want to have a look here: https://github.com/sbt/sbt/issues/6294#issuecomment-822495368

script:
  - sbt -sbt-launch-repo https://repo1.maven.org/maven2 ++$TRAVIS_SCALA_VERSION test
ziggythehamster commented 3 years ago

I have filed a support request with Travis-CI about this, hopefully they can fix it without us needing to do a workaround :)

denzhel commented 3 years ago

Another option is to create a file named .sbtopts in the root folder of your project:

.sbtopts

with the following content:

-sbt-launch-repo
https://repo1.maven.org/maven2

Thanks to @yahel2410

SethTisue commented 3 years ago

is anyone finding that -sbt-launch-repo https://repo1.maven.org/maven2 actually remains necessary? according to @JustinPihony at https://github.com/sbt/sbt/issues/6294#issuecomment-824305454 , the right redirects are supposed to be in place, as of yesterday, to make it not necessary anymore

denzhel commented 3 years ago

is anyone finding that -sbt-launch-repo https://repo1.maven.org/maven2 actually remains necessary? according to @JustinPihony at sbt/sbt#6294 (comment) , the right redirects are supposed to be in place, as of yesterday, to make it not necessary anymore

I tested it on a build that failed before and now it finishes successfully.

eed3si9n commented 3 years ago

Here's a radical idea. Why don't we stop using fake Bash script, and use the official sbt runner script maintained by the sbt project?

install:
  - |
    export SBT_OPTS="-Dfile.encoding=UTF-8"
    curl -L --silent "https://raw.githubusercontent.com/sbt/sbt/v1.5.1/sbt" > $HOME/sbt
    chmod +x $HOME/sbt && sudo mv $HOME/sbt /usr/local/bin/sbt
dwijnand commented 3 years ago

Not sure what makes the script fake, but it was probably chosen by Travis CI because it's lightweight and can reliably run all versions of sbt by downloading the necessary launcher jar.

Good to see that design has been adopted by the official script.

eed3si9n commented 3 years ago

The idea of downloading launcher JAR was inspired by sbt-extras.

This is probably a tangential point to everyone except for @dwijnand, but note that sbt 1.5.1 runner will download the sbt-launch 1.5.1 (similar to grabbing sbt-1.5.1.tar.gz) regardless of the sbt version in project/build.properties because sbt-launch is already capable of launching all versions of sbt.