Spack's default timeout is too short

robertu94 commented 4 years ago

Spack occasionally aborts a fetch too early when a url source is too slow. I ran into this issue with util-macros on a busy night.

Steps to reproduce the issue

$ spack install util-macros

Steps that resolved the issue (workaround)

I edited spack/lib/spack/spack/fetch_strategy.py to increase the timeout to 1 minute from 10 seconds. Ideally this should be configurable.

Error Message

==> Installing util-macros
==> Searching for binary cache of util-macros
==> No binary for util-macros found: installing from source
==> Fetching https://www.x.org/archive/individual/util/util-macros-1.19.1.tar.bz2

curl: (28) Operation timed out after 10001 milliseconds with 0 out of 0 bytes received
==> Failed to fetch file from URL: https://www.x.org/archive/individual/util/util-macros-1.19.1.tar.bz2
    Curl failed with error 28
==> Fetching from https://www.x.org/archive/individual/util/util-macros-1.19.1.tar.bz2 failed.
==> Error: FetchError: All fetchers failed for spack-stage-util-macros-1.19.1-3khni2zc7qm6ruqeot3pbstb2lb36uhd

/blues/gpfs/home/underwoo/git/spack/lib/spack/spack/package.py:1110, in do_fetch:
       1107                raise FetchError("Will not fetch %s" %
       1108                                 self.spec.format('{name}{@version}'), ck_msg)
       1109
  >>   1110        self.stage.create()
       1111        self.stage.fetch(mirror_only)
       1112        self._fetch_time = time.time() - start_time
       1113

Information on your system

I ran this on ANL's bebop, but also replicated it on Clemson's palmetto using spack's default configuration.

tjjjohnson commented 4 years ago

I also had this issue with libbsd today. In my case it worked on about the 10th try.

scemama commented 4 years ago

I also had this issue today with libffi and bzip2.

alalazo commented 4 years ago

I also had this issue today with libffi and bzip2.

For the record build tests on develop at https://github.com/spack/spack/commit/8d8925c725491cf028ef6c1b93e431dcb200f439 were failing because of that.

michaelkuhn commented 4 years ago

I just ran into the same problem and created a way to configure the timeout in PR #15212. Let me know whether this works for you as well.

spack / spack