Linuxbrew / legacy-linuxbrew

:skull: This repository is defunct, because it has been split into https://github.com/Linuxbrew/brew and https://github.com/Linuxbrew/homebrew-core
http://linuxbrew.sh
Other
2.23k stars 295 forks source link

[openssl] failed on Ubuntu 14.04.4 LTS #1053

Closed channprj closed 8 years ago

channprj commented 8 years ago

Thank you for any help in advance. I closely examine the issues, but I can't find solution.

I'm going to install pyenv or pip, but brew doesn't work. (I installed it another way, but I want to use it on brew.)

Logs are here:

==> Downloading https://www.openssl.org/source/openssl-1.0.2g.tar.gz
Already downloaded: /home/channprj/.cache/Homebrew/openssl-1.0.2g.tar.gz
==> Downloading https://raw.githubusercontent.com/Homebrew/patches/3f1dc8ea145a70543aded8101a0c725abf82fc45/openssl/revert-pass-pure-constants-verbatim.patch
Already downloaded: /home/channprj/.cache/Homebrew/openssl--patch-e38f84181a56e70028ade8408ad70aaffaea386b7e1b35de55728ae878d544aa.patch
==> Downloading https://raw.githubusercontent.com/Homebrew/patches/3f1dc8ea145a70543aded8101a0c725abf82fc45/openssl/tshort-asm.patch
Already downloaded: /home/channprj/.cache/Homebrew/openssl--patch-f161e2fc1395efcb53d785004d67d4962d28aa8ce282a91020f12809c03b2afd.patch
==> Patching
==> Applying revert-pass-pure-constants-verbatim.patch
patching file crypto/perlasm/x86_64-xlate.pl
==> Applying tshort-asm.patch
patching file crypto/perlasm/x86_64-xlate.pl
==> perl ./Configure --prefix=/home/channprj/.linuxbrew/Cellar/openssl/1.0.2g --openssldir=/home/channprj/.linuxbrew/etc/openssl no-ssl2 zlib-dynamic shared enable-cms -isystem/home/channprj/.linuxbrew/include -Os -w -pipe -march=core2 -L/home/channprj/.linuxbrew/lib -Wl,--dynamic-linker=/home/channprj/.linuxbrew/lib/ld.so -Wl,-rpath,/home/channprj/.linuxbrew/lib linux-x86_64
==> make depend
==> make
==> make test
Last 15 lines from /home/channprj/.cache/Homebrew/Logs/openssl/04.make:
test sslv2
Testing was requested for a disabled protocol. Skipping tests.
test sslv2 with server authentication
Testing was requested for a disabled protocol. Skipping tests.
test sslv2 with client authentication
Testing was requested for a disabled protocol. Skipping tests.
test sslv2 with both client and server authentication
Testing was requested for a disabled protocol. Skipping tests.
test sslv3
Available compression methods:
  1: zlib compression
Illegal instruction (core dumped)
make[1]: *** [test_ssl] Error 1
make[1]: Leaving directory `/tmp/openssl20160407-2772-vcvgsa/openssl-1.0.2g/test'
make: *** [tests] Error 2

READ THIS: https://github.com/Linuxbrew/linuxbrew/blob/master/share/doc/homebrew/Troubleshooting.md#troubleshooting

These open issues may also help:
openssl: ../util/domd: makedepend: not found https://github.com/Linuxbrew/linuxbrew/issues/944
openssl upgrade fails if Linuxbrew cURL installed https://github.com/Linuxbrew/linuxbrew/issues/883
Ubuntu 10.04 openssl installation fails https://github.com/Linuxbrew/linuxbrew/issues/657

...and this is my system version.

Distributor ID: Ubuntu
Description:    Ubuntu 14.04.4 LTS
Release:    14.04
Codename:   trusty
channprj@ubt-chann:~/Desktop » sudo apt-get install openssl
Reading package lists... Done
Building dependency tree       
Reading state information... Done
openssl is already the newest version.
0 upgraded, 0 newly installed, 0 to remove and 5 not upgraded.

and specific logs are on the gist. (Lots of lines indicate missing items in gist, but that items are in my computer.)

sjackman commented 8 years ago

Illegal instruction (core dumped)

I believe your CPU does not support an instruction used in one of the precompiled bottles, possibly zlib.

CPU: quad-core 64-bit 0x1005

Try

brew reinstall zlib -s
brew install openssl

This issue will likely come up again with other packages. You'll probably want to set

export HOMEBREW_BUILD_FROM_SOURCE=1
sjackman commented 8 years ago

Do you know what make and model of CPU you have?

sjackman commented 8 years ago

brew doctor should check for an unknown CPU model and report it.

channprj commented 8 years ago

Thank you for your advice, but it doesn't work. Same errors occur.

channprj@ubt-chann:~ » brew doctor
Your system is ready to brew.

and... my CPU is here.

processor   : 0
vendor_id   : AuthenticAMD
cpu family  : 16
model       : 5
model name  : AMD Athlon(tm) II X4 645 Processor
stepping    : 3
microcode   : 0x10000c8
cpu MHz     : 3100.000
cache size  : 512 KB
physical id : 0
siblings    : 4
core id     : 0
cpu cores   : 4
apicid      : 0
initial apicid  : 0
fpu     : yes
fpu_exception   : yes
cpuid level : 5
wp      : yes
flags       : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm 3dnowext 3dnow constant_tsc rep_good nopl nonstop_tsc extd_apicid pni monitor cx16 popcnt lahf_lm cmp_legacy svm extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw ibs skinit wdt hw_pstate npt lbrv svm_lock nrip_save vmmcall
bugs        : tlb_mmatch fxsave_leak sysret_ss_attrs
bogomips    : 6228.11
TLB size    : 1024 4K pages
clflush size    : 64
cache_alignment : 64
address sizes   : 48 bits physical, 48 bits virtual
power management: ts ttp tm stc 100mhzsteps hwpstate

processor   : 1
vendor_id   : AuthenticAMD
cpu family  : 16
model       : 5
model name  : AMD Athlon(tm) II X4 645 Processor
stepping    : 3
microcode   : 0x10000c8
cpu MHz     : 2400.000
cache size  : 512 KB
physical id : 0
siblings    : 4
core id     : 1
cpu cores   : 4
apicid      : 1
initial apicid  : 1
fpu     : yes
fpu_exception   : yes
cpuid level : 5
wp      : yes
flags       : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm 3dnowext 3dnow constant_tsc rep_good nopl nonstop_tsc extd_apicid pni monitor cx16 popcnt lahf_lm cmp_legacy svm extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw ibs skinit wdt hw_pstate npt lbrv svm_lock nrip_save vmmcall
bugs        : tlb_mmatch fxsave_leak sysret_ss_attrs
bogomips    : 6228.11
TLB size    : 1024 4K pages
clflush size    : 64
cache_alignment : 64
address sizes   : 48 bits physical, 48 bits virtual
power management: ts ttp tm stc 100mhzsteps hwpstate

processor   : 2
vendor_id   : AuthenticAMD
cpu family  : 16
model       : 5
model name  : AMD Athlon(tm) II X4 645 Processor
stepping    : 3
microcode   : 0x10000c8
cpu MHz     : 800.000
cache size  : 512 KB
physical id : 0
siblings    : 4
core id     : 2
cpu cores   : 4
apicid      : 2
initial apicid  : 2
fpu     : yes
fpu_exception   : yes
cpuid level : 5
wp      : yes
flags       : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm 3dnowext 3dnow constant_tsc rep_good nopl nonstop_tsc extd_apicid pni monitor cx16 popcnt lahf_lm cmp_legacy svm extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw ibs skinit wdt hw_pstate npt lbrv svm_lock nrip_save vmmcall
bugs        : tlb_mmatch fxsave_leak sysret_ss_attrs
bogomips    : 6228.11
TLB size    : 1024 4K pages
clflush size    : 64
cache_alignment : 64
address sizes   : 48 bits physical, 48 bits virtual
power management: ts ttp tm stc 100mhzsteps hwpstate

processor   : 3
vendor_id   : AuthenticAMD
cpu family  : 16
model       : 5
model name  : AMD Athlon(tm) II X4 645 Processor
stepping    : 3
microcode   : 0x10000c8
cpu MHz     : 800.000
cache size  : 512 KB
physical id : 0
siblings    : 4
core id     : 3
cpu cores   : 4
apicid      : 3
initial apicid  : 3
fpu     : yes
fpu_exception   : yes
cpuid level : 5
wp      : yes
flags       : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm 3dnowext 3dnow constant_tsc rep_good nopl nonstop_tsc extd_apicid pni monitor cx16 popcnt lahf_lm cmp_legacy svm extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw ibs skinit wdt hw_pstate npt lbrv svm_lock nrip_save vmmcall
bugs        : tlb_mmatch fxsave_leak sysret_ss_attrs
bogomips    : 6228.11
TLB size    : 1024 4K pages
clflush size    : 64
cache_alignment : 64
address sizes   : 48 bits physical, 48 bits virtual
power management: ts ttp tm stc 100mhzsteps hwpstate

Thank you for your help.

sjackman commented 8 years ago

The default optimization flags are -march=core2 -msse4, which are inappropriate for your CPU. See https://github.com/Linuxbrew/linuxbrew/blob/master/Library/Homebrew/extend/ENV/std.rb#L11 You can either change that value, or you can add a line to OPTIMIZATION_FLAGS in Library/Homebrew/os/linux/hardware.rb. See https://github.com/Linuxbrew/linuxbrew/blob/master/Library/Homebrew/os/linux/hardware.rb#L6

Try this untested patch.

diff --git a/Library/Homebrew/os/linux/hardware.rb b/Library/Homebrew/os/linux/hardware.rb
index 8e44693..cee0ff0 100644
--- a/Library/Homebrew/os/linux/hardware.rb
+++ b/Library/Homebrew/os/linux/hardware.rb
@@ -3,6 +3,7 @@ module LinuxCPUs
     :penryn => "-march=core2 -msse4.1",
     :core2 => "-march=core2",
     :core => "-march=prescott",
+    :"0x1005" => "",
     :arm => "",
   }.freeze
   def optimization_flags
alekseysidorov commented 8 years ago

The issue is also affected for me.

Architecture:          x86_64
CPU op-mode(s):        32-bit, 64-bit
Byte Order:            Little Endian
CPU(s):                6
On-line CPU(s) list:   0-5
Thread(s) per core:    1
Core(s) per socket:    6
Socket(s):             1
NUMA node(s):          1
Vendor ID:             AuthenticAMD
CPU family:            16
Model:                 10
Model name:            AMD Phenom(tm) II X6 1090T Processor
Stepping:              0
CPU MHz:               800.000
CPU max MHz:           3200,0000
CPU min MHz:           800,0000
BogoMIPS:              6428.03
Virtualization:        AMD-V
L1d cache:             64K
L1i cache:             64K
L2 cache:              512K
L3 cache:              6144K
NUMA node0 CPU(s):     0-5
Flags:                 fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm 3dnowext 3dnow constant_tsc rep_good nopl nonstop_tsc extd_apicid aperfmperf pni monitor cx16 popcnt lahf_lm cmp_legacy svm extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw ibs skinit wdt cpb hw_pstate npt lbrv svm_lock nrip_save pausefilter vmmcall

In my opinion it need to add proper Amd support although Apple does not like this hardware. Phenom processors are not too old and slow.

channprj commented 8 years ago

>> See Gist

...
     :core => "-march=prescott",
+    :"0x1005" => "",
     :arm => "",
...

I tested your new patch after $ brew update; brew doctor, but It's same as before. I'm going to find the cause of error... :sob:. Thank you for your advise.

sjackman commented 8 years ago

You'll have to remove all installed packages and start from scratch.

brew remove `brew list`
export HOMEBREW_BUILD_FROM_SOURCE=1
brew install openssl -v

I'm not positive that my above patch is exactly right. I haven't tested it. Try

diff --git a/Library/Homebrew/extend/ENV/std.rb b/Library/Homebrew/extend/ENV/std.rb
index 037c8bb..36122b2 100644
--- a/Library/Homebrew/extend/ENV/std.rb
+++ b/Library/Homebrew/extend/ENV/std.rb
@@ -8,7 +8,7 @@ module Stdenv

   # @private
   SAFE_CFLAGS_FLAGS = "-w -pipe"
-  DEFAULT_FLAGS = "-march=core2 -msse4"
+  DEFAULT_FLAGS = ""

   def self.extended(base)
     unless ORIGINAL_PATHS.include? HOMEBREW_PREFIX/"bin"
channprj commented 8 years ago

>> See Gist

Error: openssl 1.0.2g did not build

...I do not buy AMD cpu next time.. :disappointed:

sjackman commented 8 years ago

Same error.

test sslv3
Available compression methods:
  1: zlib compression
Illegal instruction (core dumped)
make[1]: *** [test_ssl] Error 1
make[1]: Leaving directory `/tmp/openssl20160411-9788-1n1lsgw/openssl-1.0.2g/test'
make: *** [tests] Error 2

Sorry, I'm running out of ideas.

There are two workarounds

brew install openssl --without-test

You can try disabling zlib for openssl.

diff --git a/Formula/openssl.rb b/Formula/openssl.rb
index 3c57f5a..3fca180 100644
--- a/Formula/openssl.rb
+++ b/Formula/openssl.rb
@@ -46,7 +46,7 @@ class Openssl < Formula
     --prefix=#{prefix}
     --openssldir=#{openssldir}
     no-ssl2
-    zlib-dynamic
+    no-zlib
     shared
     enable-cms
   ]
alekseysidorov commented 8 years ago

no-zlib does not works for me.

Available compression methods:
  NONE
Illegal instruction
Makefile:298: ошибка выполнения рецепта для цели «test_ssl»
make[1]: *** [test_ssl] Ошибка 1
make[1]: выход из каталога «/tmp/openssl20160415-11542-n9qiy0/openssl-1.0.2g/test»
Makefile:460: ошибка выполнения рецепта для цели «tests»
make: *** [tests] Ошибка 2
==> Formula
Path: /home/aleksey/.linuxbrew/Library/Formula/openssl.rb
==> Configuration
HOMEBREW_VERSION: 0.9.5
ORIGIN: https://github.com/Homebrew/linuxbrew
HEAD: 284ee0ef39216b2c7b1e043b5df233c717bbf9a8
Last commit: 9 минут назад
HOMEBREW_PREFIX: /home/aleksey/.linuxbrew
HOMEBREW_REPOSITORY: /home/aleksey/.linuxbrew
HOMEBREW_CELLAR: /home/aleksey/.linuxbrew/Cellar
HOMEBREW_BOTTLE_DOMAIN: https://linuxbrew.bintray.com
CPU: 6-core 64-bit phenom
Kernel: Linux 4.4.0-1-amd64 x86_64 GNU/Linux
OS: Debian GNU/Linux unstable (sid)
Codename: sid
OS glibc: 2.22
OS gcc: 5.3.1
Linuxbrew glibc: N/A
Linuxbrew gcc: N/A
Clang: 3.7 build 0
X11: 2.7.8 => /usr
System Ruby: 2.3.0 => /usr/bin/ruby2.3
Perl: /usr/bin/perl
Python: /usr/bin/python => /usr/bin/python2.7
Ruby: /usr/bin/ruby => /usr/bin/ruby2.3
Java: Picked up _JAVA_OPTIONS: -Dawt.useSystemAAFontSettings=gasp
==> ENV
CC: /usr/bin/gcc-5
CXX: /usr/bin/g++-5
OBJC: /usr/bin/gcc-5
OBJCXX: /usr/bin/g++-5
CFLAGS: -Os -w -pipe -march=core2
CXXFLAGS: -Os -w -pipe -march=core2
CPPFLAGS: -isystem/home/aleksey/.linuxbrew/include
LDFLAGS: -L/home/aleksey/.linuxbrew/lib -Wl,--dynamic-linker=/home/aleksey/.linuxbrew/lib/ld.so -Wl,-rpath,/home/aleksey/.linuxbrew/lib
CMAKE_PREFIX_PATH: /home/aleksey/.linuxbrew
PKG_CONFIG_LIBDIR: /home/aleksey/.linuxbrew/lib/pkgconfig
HOMEBREW_BUILD_FROM_SOURCE: 1
PATH: /home/aleksey/.linuxbrew/bin:/home/aleksey/.cargo/bin:/home/aleksey/.linuxbrew/sbin:/home/aleksey/Applications/Android/sdk/platform-tools:/usr/local/bin:/usr/bin:/bsr/local/games:/usr/games:/home/aleksey/.linuxbrew/Library/ENV/scm
CPATH: /home/aleksey/.linuxbrew/include
LD_LIBRARY_PATH: /home/aleksey/.linuxbrew/Cellar/openssl/1.0.2g/lib:/home/aleksey/.linuxbrew/Cellar/openssl/1.0.2g
LD_RUN_PATH: /home/aleksey/.linuxbrew/lib
LIBRARY_PATH: /home/aleksey/.linuxbrew/lib

Error: openssl 1.0.2g did not build

I always thought that mach=core2 is enough for amd k10 family.

sjackman commented 8 years ago

If you brew install openssl --without-test does brew test openssl work?

sjackman commented 8 years ago

If you know how to use gdb, you could determine which instruction (opcode) is causing the failure to help debugging.

ulimit -c unlimited
brew install openssl -dv
# select shell
gdb test-executable core-file
x/i

Change test-executable and core-file to the appropriate file names.

alekseysidorov commented 8 years ago

I did not find the file that causes the fall in openssl tests, but I found illegal instruction in python.

0x7ffff6478b36 <strop_maketrans+550>    pshufb %xmm5,%xmm2  

It seems like k10 does not support SSSE3. This is similar to the bug report core.27879.zip

sjackman commented 8 years ago

The default optimization flags are DEFAULT_FLAGS = "-march=core2 -msse4". See above for suggestions on how to change either DEFAULT_FLAGS or to change the optimization flags specifically for your processor.

tseemann commented 8 years ago

@alekseysidorov this site may be helpful: https://wiki.gentoo.org/wiki/Safe_CFLAGS

@sjackman I thought -march=native will auto-detect your CPU and even choose appropriate SSE/AVX flags?

@sjackman I've just discovered I can use -march=core-avx2 for my Haswell Xeons. Are there any env vars we can set that affect ENV.std ?

sjackman commented 8 years ago

@sjackman I thought -march=native will auto-detect your CPU and even choose appropriate SSE/AVX flags?

Yes, it does. Thanks for the reminder. The default CFLAGS should definitely be -march=native. The default bottle flags should continue to be -march=core2.

Homebrew superenv uses -march=native, but stdenv (which Linuxbrew uses) does not. The historical reason was that GCC 4.2 which was used by Mac OS before clang was the default compiler, didn't support -march=native. See https://github.com/Homebrew/legacy-homebrew/issues/14204

sjackman commented 8 years ago

@sjackman I've just discovered I can use -march=core-avx2 for my Haswell Xeons. Are there any env vars we can set that affect ENV.std ?

No, not that I'm aware of. My suggestion is to change DEFAULT_FLAGS to -march=native. What does brew config |grep CPU report on your machine?

sjackman commented 8 years ago

-march=native was added to GCC in 4.2, so we can safely use it. https://gcc.gnu.org/gcc-4.2/changes.html Apple GCC 4.2 must not have had it, I guess.

alekseysidorov commented 8 years ago

For unknown reasons linuxbrew interprets phenom II as core2 processor family. If I replace

:core2 => "-march=core2",

by

:core2 => "-march=native",

compiling is finished without errors.

sjackman commented 8 years ago

That code lives here, if you want to troubleshoot it: https://github.com/Linuxbrew/linuxbrew/blob/master/Library/Homebrew/os/linux/hardware.rb#L59

alekseysidorov commented 8 years ago

In my opinion the problem is there: https://github.com/Linuxbrew/linuxbrew/blob/master/Library/Homebrew/extend/ENV/std.rb#L368 On 64 bit oldest cpu means core2 https://github.com/Linuxbrew/linuxbrew/blob/master/Library/Homebrew/hardware.rb#L76

sjackman commented 8 years ago

Yep, that's the problem, and this line in particular:

    elsif Hardware::CPU.intel? && !Hardware::CPU.sse4?

https://github.com/Linuxbrew/linuxbrew/blob/master/Library/Homebrew/extend/ENV/std.rb#L364

That line should be disable on Linux. Try:

    elsif OS.mac? && Hardware::CPU.intel? && !Hardware::CPU.sse4?
sjackman commented 8 years ago

If that works for you, please submit a pull request.

alekseysidorov commented 8 years ago

https://github.com/Linuxbrew/linuxbrew/pull/1094

sjackman commented 8 years ago

@channprj Does pull request #1094 fix your issue?