uwsampa / grappa

Grappa: scaling irregular applications on commodity clusters
grappa.io
BSD 3-Clause "New" or "Revised" License
159 stars 50 forks source link

build broken by http://gflags.googlecode.com/files/gflags-2.0-no-svn-files.tar.gz download issues #199

Closed jeffhammond closed 9 years ago

jeffhammond commented 9 years ago

Can you provide a way for me to download http://gflags.googlecode.com/files/gflags-2.0-no-svn-files.tar.gz myself and point the build system to it? I am behind a firewall and despite having all of the proxy settings necessary to interact with outside websites via git, hg, ssh, svn and wget, this apparently does not fix the problem.

And I can access http://gflags.googlecode.com/files/gflags-2.0-no-svn-files.tar.gz via both wget and curl, but only wget does the right thing. I don't know what's wrong with curl, or how to tell CMake to not use it and instread try wget.

Thanks,

Jeff

jrhammon-mac01:github jrhammon$ CC=/opt/mpich/dev/intel/fast/bin/mpicc CXX=/opt/mpich/dev/intel/fast/bin/mpicxx ./configure 
cmake /Users/jrhammon/Work/PGAS/GRAPPA/github -G"Unix Makefiles" -DSHMMAX=4194304 -DCMAKE_C_COMPILER=/opt/mpich/dev/intel/fast/bin/mpicc -DCMAKE_CXX_COMPILER=/opt/mpich/dev/intel/fast/bin/mpicxx -DBASE_C_COMPILER=/opt/mpich/dev/intel/fast/bin/mpicc -DBASE_CXX_COMPILER=/opt/mpich/dev/intel/fast/bin/mpicxx -DCMAKE_BUILD_TYPE=RelWithDebInfo 
-- The C compiler identification is Intel 15.0.1.20141022
-- The CXX compiler identification is Intel 15.0.1.20141022
-- Check for working C compiler: /opt/mpich/dev/intel/fast/bin/mpicc
-- Check for working C compiler: /opt/mpich/dev/intel/fast/bin/mpicc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working CXX compiler: /opt/mpich/dev/intel/fast/bin/mpicxx
-- Check for working CXX compiler: /opt/mpich/dev/intel/fast/bin/mpicxx -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Boost found:  1.55.0 -- /usr/local
-- Found MPI_C: /opt/mpich/dev/gcc/default/lib/libmpi.dylib;/opt/mpich/dev/gcc/default/lib/libpmpi.dylib  
-- Found MPI_CXX: /opt/mpich/dev/gcc/default/lib/libmpicxx.dylib;/opt/mpich/dev/gcc/default/lib/libmpi.dylib;/opt/mpich/dev/gcc/default/lib/libpmpi.dylib  
-- Using {/opt/mpich/dev/intel/fast/bin/mpicc,/opt/mpich/dev/intel/fast/bin/mpicxx} to build third-party dependencies (if needed).
     gflags: <building>
     glog:   <building>
-- The ASM compiler identification is Intel
-- Found assembler: /opt/mpich/dev/intel/fast/bin/mpicc
-- Found Doxygen: /usr/local/bin/doxygen (found version "1.8.8") 
-- Configuring done
-- Generating done
-- Build files have been written to: /Users/jrhammon/Work/PGAS/GRAPPA/github/build/Make+Release
-------------------------------------
created build/Make+Release; to build:
> cd build/Make+Release; make -j
-------------------------------------
jrhammon-mac01:github jrhammon$ cd build/Make+Release; make
Scanning dependencies of target third-party-gflags
[  2%] Creating directories for 'third-party-gflags'
[  2%] Performing download step (download, verify and extract) for 'third-party-gflags'
-- downloading...
     src='http://gflags.googlecode.com/files/gflags-2.0-no-svn-files.tar.gz'
     dst='/Users/jrhammon/Work/PGAS/GRAPPA/github/build/Make+Release/third-party/build/gflags/src/gflags-2.0-no-svn-files.tar.gz'
     timeout='none'
CMake Error at /Users/jrhammon/Work/PGAS/GRAPPA/github/build/Make+Release/third-party/build/stamps/gflags/download-third-party-gflags.cmake:22 (message):
  error: downloading
  'http://gflags.googlecode.com/files/gflags-2.0-no-svn-files.tar.gz' failed

    status_code: 7
    status_string: "Couldn't connect to server"
    log: Adding handle: conn: 0x7f99a2813600

  Adding handle: send: 0

  Adding handle: recv: 0

  Curl_addHandleToPipeline: length: 1

  - Conn 0 (0x7f99a2813600) send_pipe: 1, recv_pipe: 0

  About to connect() to gflags.googlecode.com port 80 (#0)

    Trying 74.125.20.82...
    Trying 2607:f8b0:4002:c09::52...

  Failed to connect to 2607:f8b0:4002:c09::52: No route to host

  Failed connect to gflags.googlecode.com:80; No route to host

  Closing connection 0

make[2]: *** [third-party/build/stamps/gflags/third-party-gflags-download] Error 1
make[1]: *** [third-party/CMakeFiles/third-party-gflags.dir/all] Error 2
make: *** [all] Error 2
nelsonje commented 9 years ago

Yep, Rob and I talked about this problem yesterday. It's easy to work around; I'll provide a mechanism to drop in predownloaded files.

On Fri, Jan 9, 2015 at 1:47 PM, Jeff Hammond notifications@github.com wrote:

Can you provide a way for me to download http://gflags.googlecode.com/files/gflags-2.0-no-svn-files.tar.gz myself and point the build system to it? I am behind a firewall and despite having all of the proxy settings necessary to interact with outside websites via git, hg, ssh, svn and wget, this apparently does not fix the problem.

And I can access http://gflags.googlecode.com/files/gflags-2.0-no-svn-files.tar.gz via both wget and curl, but only wget does the right thing. I don't know what's wrong with curl, or how to tell CMake to not use it and instread try wget.

Thanks,

Jeff

jrhammon-mac01:github jrhammon$ CC=/opt/mpich/dev/intel/fast/bin/mpicc CXX=/opt/mpich/dev/intel/fast/bin/mpicxx ./configure cmake /Users/jrhammon/Work/PGAS/GRAPPA/github -G"Unix Makefiles" -DSHMMAX=4194304 -DCMAKE_C_COMPILER=/opt/mpich/dev/intel/fast/bin/mpicc -DCMAKE_CXX_COMPILER=/opt/mpich/dev/intel/fast/bin/mpicxx -DBASE_C_COMPILER=/opt/mpich/dev/intel/fast/bin/mpicc -DBASE_CXX_COMPILER=/opt/mpich/dev/intel/fast/bin/mpicxx -DCMAKE_BUILD_TYPE=RelWithDebInfo -- The C compiler identification is Intel 15.0.1.20141022 -- The CXX compiler identification is Intel 15.0.1.20141022 -- Check for working C compiler: /opt/mpich/dev/intel/fast/bin/mpicc -- Check for working C compiler: /opt/mpich/dev/intel/fast/bin/mpicc -- works -- Detecting C compiler ABI info -- Detecting C compiler ABI info - done -- Check for working CXX compiler: /opt/mpich/dev/intel/fast/bin/mpicxx -- Check for working CXX compiler: /opt/mpich/dev/intel/fast/bin/mpicxx -- works -- Detecting CXX compiler ABI info -- Detecting CXX compiler ABI info - done -- Boost found: 1.55.0 -- /usr/local -- Found MPI_C: /opt/mpich/dev/gcc/default/lib/libmpi.dylib;/opt/mpich/dev/gcc/default/lib/libpmpi.dylib -- Found MPI_CXX: /opt/mpich/dev/gcc/default/lib/libmpicxx.dylib;/opt/mpich/dev/gcc/default/lib/libmpi.dylib;/opt/mpich/dev/gcc/default/lib/libpmpi.dylib -- Using {/opt/mpich/dev/intel/fast/bin/mpicc,/opt/mpich/dev/intel/fast/bin/mpicxx} to build third-party dependencies (if needed). gflags: glog: -- The ASM compiler identification is Intel -- Found assembler: /opt/mpich/dev/intel/fast/bin/mpicc -- Found Doxygen: /usr/local/bin/doxygen (found version "1.8.8") -- Configuring done -- Generating done

-- Build files have been written to: /Users/jrhammon/Work/PGAS/GRAPPA/github/build/Make+Release

created build/Make+Release; to build:

cd build/Make+Release; make -j

jrhammon-mac01:github jrhammon$ cd build/Make+Release; make Scanning dependencies of target third-party-gflags [ 2%] Creating directories for 'third-party-gflags' [ 2%] Performing download step (download, verify and extract) for 'third-party-gflags' -- downloading... src='http://gflags.googlecode.com/files/gflags-2.0-no-svn-files.tar.gz' dst='/Users/jrhammon/Work/PGAS/GRAPPA/github/build/Make+Release/third-party/build/gflags/src/gflags-2.0-no-svn-files.tar.gz' timeout='none' CMake Error at /Users/jrhammon/Work/PGAS/GRAPPA/github/build/Make+Release/third-party/build/stamps/gflags/download-third-party-gflags.cmake:22 (message): error: downloading 'http://gflags.googlecode.com/files/gflags-2.0-no-svn-files.tar.gz' failed

status_code: 7
status_string: "Couldn't connect to server"
log: Adding handle: conn: 0x7f99a2813600

Adding handle: send: 0

Adding handle: recv: 0

Curl_addHandleToPipeline: length: 1

  • Conn 0 (0x7f99a2813600) send_pipe: 1, recv_pipe: 0

    About to connect() to gflags.googlecode.com port 80 (#0)

    Trying 74.125.20.82... Trying 2607:f8b0:4002:c09::52...

    Failed to connect to 2607:f8b0:4002:c09::52: No route to host

    Failed connect to gflags.googlecode.com:80; No route to host

    Closing connection 0

make[2]: * [third-party/build/stamps/gflags/third-party-gflags-download] Error 1 make[1]: * [third-party/CMakeFiles/third-party-gflags.dir/all] Error 2 make: *\ [all] Error 2

— Reply to this email directly or view it on GitHub https://github.com/uwsampa/grappa/issues/199.

nelsonje commented 9 years ago

Okay, I've added a configure flag to use pre-downloaded dependences. Here's what to do:

  1. update your copy of the repo.
  2. download this file on a machine with internet connectivity: http://grappa.cs.washington.edu/files/grappa-third-party-downloads.tar
  3. go to third-party/downloads and untar the file there.
  4. go back to the root of the client and blow away any old build directories.
  5. Run configure, including the --no-downloads flag with any other flags you need.

After that, the build process should use the pre-downloaded dependencies instead of trying to access the web.