Bo98
commented
4 years ago Interestingly, I can't reproduce this on my macOS install or on the homebrew/brew docker container.
Closed rmNULL closed 4 years ago
Bo98
commented
4 years ago Interestingly, I can't reproduce this on my macOS install or on the homebrew/brew docker container.
rmNULL
commented
4 years ago Interestingly, I can't reproduce this on my macOS install or on the
homebrew/brewdocker container.
the docker debian image is fresh one. Does that not count this as a bug ?
Bo98
commented
4 years ago Oh, I'm not saying it's not a bug. I'm just wondering why it's only crashing under certain setups - and what part of that setup might be triggering it.
I just tried a fresh Debian container myself with the reproduction steps you provided and also couldn't reproduce. I'll keep trying though.
rmNULL
commented
4 years ago @Bo98 you couldn't reproduce in docker image ? that's strange, does docker take anything user environment ?
Bo98
commented
4 years ago
Illegal instruction
Could it perhaps be a CPU instruction set issue? What CPU are you using?
rmNULL
commented
4 years ago HOMEBREW_VERSION: 2.4.0-10-gc120b61-dirty
ORIGIN: https://github.com/Homebrew/brew
HEAD: c120b61204b2e9883a3c4b3e2a7cfcb12314038a
Last commit: 3 hours ago
Core tap ORIGIN: https://github.com/Homebrew/linuxbrew-core
Core tap HEAD: 823e6f6c296c8c40afbfc401b7f46cdb7eaab68e
Core tap last commit: 3 hours ago
HOMEBREW_PREFIX: /home/linuxbrew/.linuxbrew
HOMEBREW_DEVELOPER: set
HOMEBREW_DISPLAY: :0.0
HOMEBREW_EDITOR: nvim
HOMEBREW_MAKE_JOBS: 4
CPU: quad-core 64-bit ivybridge
Homebrew Ruby: 2.6.3 => /home/linuxbrew/.linuxbrew/Homebrew/Library/Homebrew/vendor/portable-ruby/2.6.3_1/bin/ruby
Clang: 10.0 build (parse error)
Git: 2.27.0 => /bin/git
Curl: 7.70.0 => /usr/bin/curl
Kernel: Linux 5.4.45_1 x86_64 GNU/Linux
OS: Void Linux (void)
Host glibc: 2.30
/usr/bin/gcc: 9.3.0
glibc: N/A
gcc: N/A
xorg: N/A
Illegal instructionCould it perhaps be a CPU instruction set issue? What CPU are you using?
x86_64
woodruffw
commented
4 years ago Could it perhaps be a CPU instruction set issue? What CPU are you using?
x86_64
@Bo98 probably means what specific sub-family of x86-64, since Homebrew only (AFAIK) supports x86-64. What does /proc/cpuinfo show?
rmNULL
commented
4 years ago @Bo98 probably means what specific sub-family of x86-64, since Homebrew only (AFAIK) supports x86-64. What does /proc/cpuinfo show?
processor : 0
vendor_id : GenuineIntel
cpu family : 6
model : 58
model name : Intel(R) Core(TM) i5-3570K CPU @ 3.40GHz
stepping : 9
microcode : 0x16
cpu MHz : 2320.315
cache size : 6144 KB
physical id : 0
siblings : 4
core id : 0
cpu cores : 4
apicid : 0
initial apicid : 0
fpu : yes
fpu_exception : yes
cpuid level : 13
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc cpuid aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm pcid sse4_1 sse4_2 popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm cpuid_fault epb pti tpr_shadow vnmi flexpriority ept vpid fsgsbase smep erms xsaveopt dtherm ida arat pln pts
bugs : cpu_meltdown spectre_v1 spectre_v2 spec_store_bypass l1tf mds swapgs itlb_multihit
bogomips : 6784.85
clflush size : 64
cache_alignment : 64
address sizes : 36 bits physical, 48 bits virtual
power management:
processor : 1
vendor_id : GenuineIntel
cpu family : 6
model : 58
model name : Intel(R) Core(TM) i5-3570K CPU @ 3.40GHz
stepping : 9
microcode : 0x16
cpu MHz : 2352.097
cache size : 6144 KB
physical id : 0
siblings : 4
core id : 1
cpu cores : 4
apicid : 2
initial apicid : 2
fpu : yes
fpu_exception : yes
cpuid level : 13
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc cpuid aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm pcid sse4_1 sse4_2 popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm cpuid_fault epb pti tpr_shadow vnmi flexpriority ept vpid fsgsbase smep erms xsaveopt dtherm ida arat pln pts
bugs : cpu_meltdown spectre_v1 spectre_v2 spec_store_bypass l1tf mds swapgs itlb_multihit
bogomips : 6784.85
clflush size : 64
cache_alignment : 64
address sizes : 36 bits physical, 48 bits virtual
power management:
processor : 2
vendor_id : GenuineIntel
cpu family : 6
model : 58
model name : Intel(R) Core(TM) i5-3570K CPU @ 3.40GHz
stepping : 9
microcode : 0x16
cpu MHz : 1890.056
cache size : 6144 KB
physical id : 0
siblings : 4
core id : 2
cpu cores : 4
apicid : 4
initial apicid : 4
fpu : yes
fpu_exception : yes
cpuid level : 13
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc cpuid aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm pcid sse4_1 sse4_2 popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm cpuid_fault epb pti tpr_shadow vnmi flexpriority ept vpid fsgsbase smep erms xsaveopt dtherm ida arat pln pts
bugs : cpu_meltdown spectre_v1 spectre_v2 spec_store_bypass l1tf mds swapgs itlb_multihit
bogomips : 6784.85
clflush size : 64
cache_alignment : 64
address sizes : 36 bits physical, 48 bits virtual
power management:
processor : 2
vendor_id : GenuineIntel
cpu family : 6
model : 58
model name : Intel(R) Core(TM) i5-3570K CPU @ 3.40GHz
stepping : 9
microcode : 0x16
cpu MHz : 1890.056
cache size : 6144 KB
physical id : 0
siblings : 4
core id : 2
cpu cores : 4
apicid : 4
initial apicid : 4
fpu : yes
fpu_exception : yes
cpuid level : 13
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc cpuid aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm pcid sse4_1 sse4_2 popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm cpuid_fault epb pti tpr_shadow vnmi flexpriority ept vpid fsgsbase smep erms xsaveopt dtherm ida arat pln pts
bugs : cpu_meltdown spectre_v1 spectre_v2 spec_store_bypass l1tf mds swapgs itlb_multihit
bogomips : 6784.85
clflush size : 64
cache_alignment : 64
address sizes : 36 bits physical, 48 bits virtual
power management:
processor : 3
vendor_id : GenuineIntel
cpu family : 6
model : 58
model name : Intel(R) Core(TM) i5-3570K CPU @ 3.40GHz
stepping : 9
microcode : 0x16
cpu MHz : 1647.517
cache size : 6144 KB
physical id : 0
siblings : 4
core id : 3
cpu cores : 4
apicid : 6
initial apicid : 6
fpu : yes
fpu_exception : yes
cpuid level : 13
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc cpuid aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm pcid sse4_1 sse4_2 popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm cpuid_fault epb pti tpr_shadow vnmi flexpriority ept vpid fsgsbase smep erms xsaveopt dtherm ida arat pln pts
bugs : cpu_meltdown spectre_v1 spectre_v2 spec_store_bypass l1tf mds swapgs itlb_multihit
bogomips : 6784.85
clflush size : 64
cache_alignment : 64
address sizes : 36 bits physical, 48 bits virtual
power management:Ok, just ran the Linux build of portable Ruby through a disassembler. Your crash is when it hits
shlx rcx, rbx, rdxThis is a part of Bit Manipulation Instruction Set 2 (BMI2) which was introduced with Haswell. Haswell is one generation newer than the Ivy Bridge you are running.
I think the Linux build is miscompiled. I'm not sure how that happened though - it should be compiled with -march=core2.
rmNULL
commented
4 years ago @Bo98 Good find, that should be fixed soon then. Thanks :1st_place_medal:
Bo98
commented
4 years ago Ok, I figured out how it happened. On Linux, the value of -march is calculated here: https://github.com/Homebrew/brew/blob/e254e804ee974db8dfea3731484cf3d7b39f1439/Library/Homebrew/extend/os/linux/extend/ENV/shared.rb
Basically:
--build-bottle and --bottle-arch is passed, use the value of --bottle-arch.--build-bottle is used without --bottle-arch, use core2 (for x86_64).:native.While core2 should be chosen for Ruby itself - it isn't for all of the dependencies as we do not build bottles for them. We run brew install -s, as seen here: https://github.com/Homebrew/homebrew-portable-ruby/blob/6e0078869cb9d2f533472f5bef7d6ea87d655728/cmd/portable-package.rb#L43-L44
This invokation will result in -march=native for all dependencies, which are later statically linked into Ruby.
rmNULL
commented
4 years ago @Bo98 Building from source in brew scenario usually means it will be used only in the build machine, using native instruction makes sense.
In vendor ruby you build on native, and distribute it to others(who may have incompatible arch as in this case). is that correct ?
sjackman
commented
4 years ago While core2 should be chosen for Ruby itself - it isn't for all of the dependencies as we do not build bottles for them.
🤦♂️ Good find, Bo, super sleuth! 🕵️♂️
sjackman
commented
4 years ago If none of the dependencies have a postinstall, the simplest solution is to replace brew install -s with brew install --build-bottle. That could be fragile if one of the dependencies later gains a necessary postinstall. Is --bottle-arch ignored in brew install -s --bottle-arch=core2 ?
Bo98
commented
4 years ago The use of (Edit: ah I see - I misunderstood. It was because we want to build the likes of pkg-config from source too to avoid using bottles rather than producing.) -s on dependencies is Linux specific. The comment by it says Avoid installing glibc. Bottles depend on glibc. Is this still a concern?
Is
--bottle-archignored inbrew install -s --bottle-arch=core2?
Yes.
In vendor ruby you build on native, and distribute it to others(who may have incompatible arch as in this case). is that correct ?
Yes. We should not be using a native -march build for that reason.
mistydemeo
commented
4 years ago Excellent catch. This is definitely a bug in brew portable-package. I'll open a PR to fix that.
Bo98
commented
4 years ago Ok it was doing brew install -s --build-bottle before, via the HOMEBREW_BUILD_BOTTLE env but that env has since been removed: https://github.com/Homebrew/brew/commit/674259d691ef2380595539d0387030892e215917. That will explain how it worked with revision 0. Adding --build-bottle is probably the easiest fix.
rmNULL
commented
4 years ago thanks @Bo98 @mistydemeo , it's fine on the user end now.
Summary
bundlecrashes after update to portable ruby 2.6.3_1Reproduction Steps
Expected Output
or not crash
Error log