Brewing Falaise/Bayeux3 on linux - openssl trouble

[cherylepatrick]

Concurrently with my laptop install, I'm also installing on the UCL batch farm, which is running Linux (Scientific Linux release 6.9 (Carbon)). I'm updating from an old and neglected copy that I installed about a year ago, and did "brew rm bayeux falaise" first.

1) I ran "brew update" and it failed; google suggested this was a known home-brew problem and that it might work if I ran it again. It appeared to work the second time. 2) I ran "brew upgrade" and it crashed after several minutes. In the output it mentioned a lot of packages it had failed to download. 3) I ran wget to get those packages and put them in .cache/Homebrew. When I ran "brew upgrade" again, it seemed like it had successfully found and installed them, but then failed on some others. I repeated this cycle (download the packages it failed on, run again, download the packages it failed on next time) for maybe 10 iterations. 4) I noticed that I started seeing a few packages fail even though I had downloaded them: readline, ninja in particular. 5) I gave up on this method and tried "brew update" again a the internet suggested this might help. It failed complaining about missing openssl certificates. It was right, the certificate it is looking for is not there. I think openssl is one of the packages brew upgrade was upgrading. 6) I tried just doing a brew install of one of the repeat-failure packages (readline) and when I did this, I also got an openssl error. 7) I tried reinstalling openssl and that failed too.

Now I am out of ideas. I'll paste the errors from the various things that don't work in separate comments so they are easier to find.

[cherylepatrick]

[cpatrick@pc133 ~]$ brew update fatal: unable to access 'https://github.com/Linuxbrew/homebrew-core/': error setting certificate verify locations: CAfile: /home/cpatrick/CadfaelBrew/etc/openssl/cert.pem fatal: unable to access 'https://github.com/SuperNEMO-DBD/brew.git/': error setting certificate verify locations: CAfile: /home/cpatrick/CadfaelBrew/etc/openssl/cert.pem CApath: /home/cpatrick/CadfaelBrew/etc/openssl/certs CApath: /home/cpatrick/CadfaelBrew/etc/openssl/certs fatal: unable to access 'https://github.com/Homebrew/homebrew-dupes/': error setting certificate verify locations: CAfile: /home/cpatrick/CadfaelBrew/etc/openssl/cert.pem CApath: /home/cpatrick/CadfaelBrew/etc/openssl/certs fatal: unable to access 'https://github.com/SuperNEMO-DBD/homebrew-cadfael/': error setting certificate verify locations: CAfile: /home/cpatrick/CadfaelBrew/etc/openssl/cert.pem CApath: /home/cpatrick/CadfaelBrew/etc/openssl/certs Error: Fetching /home/cpatrick/CadfaelBrew failed! Fetching /home/cpatrick/CadfaelBrew/Library/Taps/homebrew/homebrew-core failed! Fetching /home/cpatrick/CadfaelBrew/Library/Taps/homebrew/homebrew-dupes failed! Fetching /home/cpatrick/CadfaelBrew/Library/Taps/supernemo-dbd/homebrew-cadfael failed!

[cherylepatrick]

[cpatrick@pc133 ~]$ brew install readline Updating Homebrew... ==> Downloading https://ftp.gnu.org/gnu/readline/readline-7.0.tar.gz

curl: (77) error setting certificate verify locations: CAfile: /home/cpatrick/CadfaelBrew/etc/openssl/cert.pem CApath: /home/cpatrick/CadfaelBrew/etc/openssl/certs Trying a mirror... ==> Downloading https://ftpmirror.gnu.org/readline/readline-7.0.tar.gz

curl: (77) error setting certificate verify locations: CAfile: /home/cpatrick/CadfaelBrew/etc/openssl/cert.pem CApath: /home/cpatrick/CadfaelBrew/etc/openssl/certs Error: Failed to download resource "readline" Download failed: https://ftpmirror.gnu.org/readline/readline-7.0.tar.gz

[cherylepatrick]

[cpatrick@pc133 ~]$ brew reinstall openssl ==> Reinstalling openssl /home/cpatrick/CadfaelBrew/Cellar/ruby/2.3.1/lib/ruby/2.3.0/rubygems/core_ext/kernel_require.rb:55:in require': libcrypto.so.1.0.0: cannot open shared object file: No such file or directory - /home/cpatrick/CadfaelBrew/Cellar/ruby/2.3.1/lib/ruby/2.3.0/x86_64-linux/digest/md5.so (LoadError) from /home/cpatrick/CadfaelBrew/Cellar/ruby/2.3.1/lib/ruby/2.3.0/rubygems/core_ext/kernel_require.rb:55:inrequire' from /home/cpatrick/CadfaelBrew/Library/Homebrew/formulary.rb:1:in <top (required)>' from /home/cpatrick/CadfaelBrew/Cellar/ruby/2.3.1/lib/ruby/2.3.0/rubygems/core_ext/kernel_require.rb:55:inrequire' from /home/cpatrick/CadfaelBrew/Cellar/ruby/2.3.1/lib/ruby/2.3.0/rubygems/core_ext/kernel_require.rb:55:in require' from /home/cpatrick/CadfaelBrew/Library/Homebrew/formula.rb:9:in<top (required)>' from /home/cpatrick/CadfaelBrew/Cellar/ruby/2.3.1/lib/ruby/2.3.0/rubygems/core_ext/kernel_require.rb:55:in require' from /home/cpatrick/CadfaelBrew/Cellar/ruby/2.3.1/lib/ruby/2.3.0/rubygems/core_ext/kernel_require.rb:55:inrequire' from /home/cpatrick/CadfaelBrew/Library/Homebrew/readall.rb:1:in <top (required)>' from /home/cpatrick/CadfaelBrew/Cellar/ruby/2.3.1/lib/ruby/2.3.0/rubygems/core_ext/kernel_require.rb:55:inrequire' from /home/cpatrick/CadfaelBrew/Cellar/ruby/2.3.1/lib/ruby/2.3.0/rubygems/core_ext/kernel_require.rb:55:in require' from /home/cpatrick/CadfaelBrew/Library/Homebrew/tap.rb:2:in<top (required)>' from /home/cpatrick/CadfaelBrew/Cellar/ruby/2.3.1/lib/ruby/2.3.0/rubygems/core_ext/kernel_require.rb:55:in require' from /home/cpatrick/CadfaelBrew/Cellar/ruby/2.3.1/lib/ruby/2.3.0/rubygems/core_ext/kernel_require.rb:55:inrequire' from /home/cpatrick/CadfaelBrew/Library/Homebrew/compat/tap.rb:1:in <top (required)>' from /home/cpatrick/CadfaelBrew/Cellar/ruby/2.3.1/lib/ruby/2.3.0/rubygems/core_ext/kernel_require.rb:55:inrequire' from /home/cpatrick/CadfaelBrew/Cellar/ruby/2.3.1/lib/ruby/2.3.0/rubygems/core_ext/kernel_require.rb:55:in require' from /home/cpatrick/CadfaelBrew/Library/Homebrew/compat.rb:2:in<top (required)>' from /home/cpatrick/CadfaelBrew/Cellar/ruby/2.3.1/lib/ruby/2.3.0/rubygems/core_ext/kernel_require.rb:55:in require' from /home/cpatrick/CadfaelBrew/Cellar/ruby/2.3.1/lib/ruby/2.3.0/rubygems/core_ext/kernel_require.rb:55:inrequire' from /home/cpatrick/CadfaelBrew/Library/Homebrew/global.rb:56:in <top (required)>' from /home/cpatrick/CadfaelBrew/Cellar/ruby/2.3.1/lib/ruby/2.3.0/rubygems/core_ext/kernel_require.rb:55:inrequire' from /home/cpatrick/CadfaelBrew/Cellar/ruby/2.3.1/lib/ruby/2.3.0/rubygems/core_ext/kernel_require.rb:55:in require' from /home/cpatrick/CadfaelBrew/Library/Homebrew/build.rb:6:in

' Error: Suspicious failure

[drbenmorgan]

@cherylepatrick Could you try the steps given in this report?

https://github.com/Linuxbrew/brew/issues/341

See also:

https://github.com/Linuxbrew/legacy-linuxbrew/issues/238

if the first doesn't work.

[cherylepatrick]

As suggested in the first one I tried echo insecure >> ~/.curlrc git config --global http.sslVerify false

...and it seems to be working! Cross fingers that was the only problem, thanks!

[cherylepatrick]

Brief update on this comedy of errors - the last suggestion seemed to be going well until I used up my disk quota on the UCL linux machines. So now I am installing from scratch on a different disk with a fresh download of CadfaelBrew. All was going well until I got to the stage of trying to install falaise where I got the error: ==> Installing supernemo-dbd/cadfael/falaise dependency: icu4c ==> Downloading https://ssl.icu-project.org/files/icu4c/58.2/icu4c-58_2-src.tgz Already downloaded: /home/cpatrick/.cache/Homebrew/icu4c-58.2.tgz Error: The selected compiler doesn't support C++11: gcc-4.4

Those machines DO have an old compiler by default, so I attempted to switch to use a newer one: [cpatrick@pc133 CadfaelBrew]$ gcc --version gcc (GCC) 4.9.2 20150212 (Red Hat 4.9.2-6)

But I still get the same error complaining that gcc 4.4 is too old. Any ideas? Thanks!

[drbenmorgan]

@cherylepatrick could you check what gcc* programs you have in $(brew --prefix)/bin? Did you run brew cadfael-bootstrap as the first step after cloning?

Just to get all the info, what are the outputs of brew doctor and brew config?

[cherylepatrick]

@drbenmorgan Only 4.4 in $(brew --prefix)/bin. And yep, I did brew cadfael-bootstrap straight after cloning.

Here's the output of brew doctor - the stuff in LD_LIBRARY_PATH is from the script I ran to set the system gcc to be 4.9 rather than 4.4: `[cpatrick@pc133 CadfaelBrew]$ ./bin/brew doctor Please note that these warnings are just used to help the Homebrew maintainers with debugging if you file an issue. If everything you use Homebrew for is working fine: please don't worry and just ignore them. Thanks!

Warning: Setting LD_* vars can break dynamic linking. Set variables: LD_LIBRARY_PATH: /opt/rh/devtoolset-3/root/usr/lib64:/opt/rh/devtoolset-3/root/usr/lib

Warning: Unbrewed header files were found in /usr/local/include. If you didn't put them there on purpose they could cause problems when building Homebrew formulae, and may need to be deleted.

Unexpected header files: /usr/local/include/log_event.h /usr/local/include/trq_auth_daemon.h /usr/local/include/pbs_log.h /usr/local/include/u_lock_ctl.h /usr/local/include/setup_env.h /usr/local/include/chk_file_sec.h /usr/local/include/lib_net.h /usr/local/include/pbsnodes.h /usr/local/include/common_cmds.h /usr/local/include/attr_node_func.h /usr/local/include/lib_ifl.h /usr/local/include/lib_utils.h /usr/local/include/qsub_functions.h

Warning: You have a curlrc file If you have trouble downloading packages with Homebrew, then maybe this is the problem? If the following command doesn't work, then try removing your curlrc: curl https://github.com

Warning: Homebrew's sbin was not found in your PATH but you have installed formulae that put executables in /unix/nemo2/users/cpatrick/CadfaelBrew/sbin. Consider setting the PATH for example like so echo 'export PATH="/unix/nemo2/users/cpatrick/CadfaelBrew/sbin:$PATH"' >> ~/.bash_profile

Warning: Homebrew's share was not found in your XDG_DATA_DIRS but you have this variable set to include other locations. Some programs like vapigen may not work correctly. Consider setting the XDG_DATA_DIRS for example like so echo 'export XDG_DATA_DIRS="/unix/nemo2/users/cpatrick/CadfaelBrew/share:$XDG_DATA_DIRS"' >> ~/.bash_profile`

[cherylepatrick]

And brew config : HOMEBREW_VERSION: >1.1.0 (no git repository) ORIGIN: https://github.com/SuperNEMO-DBD/brew.git HEAD: 161e0bcdf81098f678ff177e66f813f464d3f830 Last commit: 5 weeks ago Core tap ORIGIN: https://github.com/Linuxbrew/homebrew-core Core tap HEAD: b18b61a314a41cf7c1d98bda3c1f2a84afe348c4 Core tap last commit: 3 days ago HOMEBREW_PREFIX: /unix/nemo2/users/cpatrick/CadfaelBrew HOMEBREW_REPOSITORY: /unix/nemo2/users/cpatrick/CadfaelBrew HOMEBREW_CELLAR: /unix/nemo2/users/cpatrick/CadfaelBrew/Cellar HOMEBREW_BOTTLE_DOMAIN: https://linuxbrew.bintray.com CPU: dual-core 64-bit 0xf43 Homebrew Ruby: 2.0.0-p648 => /unix/nemo2/users/cpatrick/CadfaelBrew/Library/Homebrew/vendor/portable-ruby/2.0.0-p648/bin/ruby Clang: N/A Git: 2.12.2 => /unix/nemo2/users/cpatrick/CadfaelBrew/bin/git Perl: /usr/bin/perl Python: /unix/nemo2/users/cpatrick/CadfaelBrew/bin/python => /unix/nemo2/users/cpatrick/CadfaelBrew/Cellar/python/2.7.13/bin/python2.7 Ruby: /usr/bin/ruby Java: 1.7.0_141 Kernel: Linux 2.6.32-696.el6.x86_64 x86_64 GNU/Linux OS: Scientific Linux release 6.9 (Carbon) Codename: Carbon OS glibc: 2.12 OS gcc: 4.4.7 Linuxbrew glibc: 2.19 Linuxbrew gcc: N/A Linuxbrew xorg: N/A

[cherylepatrick]

By the way /unix/nemo2/users/cpatrick/CadfaelBrew is the new location where I am attempting to install the Cadfaelbrew stuff. I tried to change over env variables to not point at the old ones (now deleted) and to point at these instead, but maybe I missed some?

[drbenmorgan]

I think this could be due to the devtoolset installing gcc/g++ without version suffices (e.g. g++-4.9). Brew, IIRC, only looks for gcc/g++ with version suffices. The cadfael-bootstrap command doesn't - it just checks the output of gcc. So what I think has happened is that the bootstrap script thinks a sufficient system GCC is available, so doesn't install the brewed gcc49, but brew install foo will only consider the gcc-4.4 link in $(brew --prefix)/bin.

There is a way to get brew to use the gcc from devtoolset-3, but, ROOT may have issues with being compiled against this (see https://root-forum.cern.ch/search?q=devtoolset%20order%3Alatest).

I can look to see if anything has changed, but in the meantime, I'd try from scratch again (sorry...) and make sure the devtoolset-3 environment isn't set up (so only the system gcc in the PATH)

[cherylepatrick]

Looks like it worked! You know what they say, fifth time's the charm... Thanks for your help and explanations, @drbenmorgan !