Closed bgruening closed 4 years ago
ping @lldelisle and @isuruf
Can you try setting the env variable OPENBLAS_CORETYPE=Nehalem
?
Can you also run lscpu
?
That worked!
(pgt) bag@bag:~/projects/code/pyGenomeTracks$ export OPENBLAS_CORETYPE=Nehalem
(pgt) bag@bag:~/projects/code/pyGenomeTracks$ python
Python 3.7.6 | packaged by conda-forge | (default, Jun 1 2020, 18:57:50)
[GCC 7.5.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import numpy as np
>>> t = np.array([[1, 0.5], [-1, 0.5]])
>>> matrix_a = np.dot([(1,2) for i in range(1000000)], t)
$ lscpu
Architektur: x86_64
CPU Operationsmodus: 32-bit, 64-bit
Byte-Reihenfolge: Little Endian
Adressgrößen: 39 bits physical, 48 bits virtual
CPU(s): 8
Liste der Online-CPU(s): 0-7
Thread(s) pro Kern: 2
Kern(e) pro Socket: 4
Sockel: 1
NUMA-Knoten: 1
Anbieterkennung: GenuineIntel
Prozessorfamilie: 6
Modell: 142
Modellname: Intel(R) Core(TM) i5-8365U CPU @ 1.60GHz
Stepping: 12
CPU MHz: 900.014
Maximale Taktfrequenz der CPU: 4100,0000
Minimale Taktfrequenz der CPU: 400,0000
BogoMIPS: 3799.90
Virtualisierung: VT-x
L1d Cache: 128 KiB
L1i Cache: 128 KiB
L2 Cache: 1 MiB
L3 Cache: 6 MiB
NUMA-Knoten0 CPU(s): 0-7
Vulnerability Itlb multihit: KVM: Mitigation: Split huge pages
Vulnerability L1tf: Not affected
Vulnerability Mds: Not affected
Vulnerability Meltdown: Not affected
Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl and seccomp
Vulnerability Spectre v1: Mitigation; usercopy/swapgs barriers and __user pointer sanitization
Vulnerability Spectre v2: Mitigation; Enhanced IBRS, IBPB conditional, RSB filling
Vulnerability Srbds: Mitigation; TSX disabled
Vulnerability Tsx async abort: Mitigation; TSX disabled
Markierungen: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc art arch_perfmon
pebs bts rep_good nopl xtopology nonstop_tsc cpuid aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid sse4_1 sse4_2 x2apic movbe p
opcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm 3dnowprefetch cpuid_fault epb invpcid_single ssbd ibrs ibpb stibp ibrs_enhanced tpr_shadow vnmi flexpriority ept vp
id ept_ad fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms invpcid mpx rdseed adx smap clflushopt intel_pt xsaveopt xsavec xgetbv1 xsaves dtherm ida arat pln pts hwp hwp_notify hwp_a
ct_window hwp_epp md_clear flush_l1d arch_capabilities
@isuruf please note that this is also crashing on Travis and at least one other computer.
Can you send a PR to revert the kernel/
changes in https://github.com/xianyi/OpenBLAS/pull/2516 ?
I can try to revert this PR. Should we pull our packages?
I'd rather not. Pulling latest will result in segfaults for others
I think the impact is rather big of this issue and its not clear how long it will take to fix it upstream. Reverting is not as easy. I will have time today in the evening to try again.
@lldelisle, what do you get with lscpu
?
Also, can you try building openblas master?
$ lscpu
Architecture: x86_64
CPU op-mode(s): 32-bit, 64-bit
Byte Order: Little Endian
Address sizes: 39 bits physical, 48 bits virtua
l
CPU(s): 4
On-line CPU(s) list: 0-3
Thread(s) per core: 2
Core(s) per socket: 2
Socket(s): 1
NUMA node(s): 1
Vendor ID: GenuineIntel
CPU family: 6
Model: 78
Model name: Intel(R) Core(TM) i7-6600U CPU @
2.60GHz
Stepping: 3
CPU MHz: 1400.015
CPU max MHz: 3400.0000
CPU min MHz: 400.0000
BogoMIPS: 5602.18
Virtualization: VT-x
L1d cache: 64 KiB
L1i cache: 64 KiB
L2 cache: 512 KiB
L3 cache: 4 MiB
NUMA node0 CPU(s): 0-3
Vulnerability Itlb multihit: KVM: Mitigation: Split huge page
s
Vulnerability L1tf: Mitigation; PTE Inversion; VMX c
onditional cache flushes, SMT vu
lnerable
Vulnerability Mds: Vulnerable: Clear CPU buffers at
tempted, no microcode; SMT vulne
rable
Vulnerability Meltdown: Mitigation; PTI
Vulnerability Spec store bypass: Vulnerable
Vulnerability Spectre v1: Mitigation; usercopy/swapgs barr
iers and __user pointer sanitiza
tion
Vulnerability Spectre v2: Mitigation; Full generic retpoli
ne, STIBP disabled, RSB filling
Vulnerability Srbds: Vulnerable: No microcode
Vulnerability Tsx async abort: Vulnerable: Clear CPU buffers at
tempted, no microcode; SMT vulne
rable
Flags: fpu vme de pse tsc msr pae mce c
x8 apic sep mtrr pge mca cmov pa
t pse36 clflush dts acpi mmx fxs
r sse sse2 ss ht tm pbe syscall
nx pdpe1gb rdtscp lm constant_ts
c art arch_perfmon pebs bts rep_
good nopl xtopology nonstop_tsc
cpuid aperfmperf pni pclmulqdq d
tes64 monitor ds_cpl vmx smx est
tm2 ssse3 sdbg fma cx16 xtpr pd
cm pcid sse4_1 sse4_2 x2apic mov
be popcnt aes xsave avx f16c rdr
and lahf_lm abm 3dnowprefetch cp
uid_fault epb invpcid_single pti
tpr_shadow vnmi flexpriority ep
t vpid ept_ad fsgsbase tsc_adjus
t bmi1 hle avx2 smep bmi2 erms i
nvpcid rtm mpx rdseed adx smap c
lflushopt intel_pt xsaveopt xsav
ec xgetbv1 xsaves dtherm ida ara
t pln pts hwp hwp_notify hwp_act
_window hwp_epp
Also, can you try building openblas master?
Can you tell me how to do that?
git clone and make?
Yes, git clone and make DYNAMIC_ARCH=1 BINARY=64 NO_LAPACK=0 TARGET=PRESCOTT NO_AFFINITY=1 USE_THREAD=1 NUM_THREADS=128 USE_OPENMP=0
and then make install PREFIX=$CONDA_PREFIX
Is this normal?
(testOpenBlas) [ldelisle@SV-49-002 OpenBLAS]$ make DYNAMIC_ARCH=1 BINARY=64 NO_LAPACK=0 TARGET=PREFIX NO_AFFINITY=1 USE_THREAD=1 NUM_THREADS=128 USE_OPENMP=0
getarch.c:1210:2: error: #error "The TARGET specified on the command line or in Makefile.rule is not supported. Please choose a target from TargetList.txt"
1210 | #error "The TARGET specified on the command line or in Makefile.rule is not supported. Please choose a target from TargetList.txt"
| ^~~~~
make: *** [Makefile.prebuild:66: getarch] Error 1
Then it continues...
Sorry, I made a mistake. Can you try, make DYNAMIC_ARCH=1 BINARY=64 NO_LAPACK=0 TARGET=PRESCOTT NO_AFFINITY=1 USE_THREAD=1 NUM_THREADS=128 USE_OPENMP=0
?
No problem, it is running with the new command.
.....
imax.c:(.text+0x10): undefined reference to `icamax_k'
collect2: error: ld returned 1 exit status
make[1]: *** [Makefile:164: cblat1] Error 1
/usr/bin/ld: ../libopenblasp-r0.3.10.dev.a(zaxpy.o): in function `zaxpy_':
zaxpy.c:(.text+0x9a): undefined reference to `zaxpy_k'
/usr/bin/ld: zaxpy.c:(.text+0xe0): undefined reference to `zaxpy_k'
/usr/bin/ld: ../libopenblasp-r0.3.10.dev.a(zswap.o): in function `zswap_':
zswap.c:(.text+0x37): undefined reference to `zswap_k'
/usr/bin/ld: ../libopenblasp-r0.3.10.dev.a(zcopy.o): in function `zcopy_':
copy.c:(.text+0x19): undefined reference to `zcopy_k'
/usr/bin/ld: copy.c:(.text+0x30): undefined reference to `zcopy_k'
/usr/bin/ld: ../libopenblasp-r0.3.10.dev.a(zscal.o): in function `zscal_':
zscal.c:(.text+0x53): undefined reference to `zscal_k'
/usr/bin/ld: zscal.c:(.text+0xa2): undefined reference to `zscal_k'
/usr/bin/ld: ../libopenblasp-r0.3.10.dev.a(zdscal.o): in function `zdscal_':
zscal.c:(.text+0x80): undefined reference to `zscal_k'
/usr/bin/ld: zscal.c:(.text+0xd1): undefined reference to `zscal_k'
/usr/bin/ld: ../libopenblasp-r0.3.10.dev.a(zdotc.o): in function `zdotc_':
zdot.c:(.text+0x19): undefined reference to `zdotc_k'
/usr/bin/ld: zdot.c:(.text+0x30): undefined reference to `zdotc_k'
/usr/bin/ld: ../libopenblasp-r0.3.10.dev.a(zdotu.o): in function `zdotu_':
zdot.c:(.text+0x19): undefined reference to `zdotu_k'
/usr/bin/ld: zdot.c:(.text+0x30): undefined reference to `zdotu_k'
/usr/bin/ld: ../libopenblasp-r0.3.10.dev.a(dzasum.o): in function `dzasum_':
asum.c:(.text+0xc): undefined reference to `zasum_k'
/usr/bin/ld: ../libopenblasp-r0.3.10.dev.a(dznrm2.o): in function `dznrm2_':
nrm2.c:(.text+0xc): undefined reference to `znrm2_k'
/usr/bin/ld: ../libopenblasp-r0.3.10.dev.a(izamax.o): in function `izamax_':
imax.c:(.text+0x10): undefined reference to `izamax_k'
collect2: error: ld returned 1 exit status
make[1]: *** [Makefile:167: zblat1] Error 1
make[1]: Leaving directory '/home/ldelisle/Documents/mygit/OpenBLAS/test'
make: *** [Makefile:140: tests] Error 2
@bgruening, @lldelisle, can you send a PR to port https://github.com/xianyi/OpenBLAS/pull/2729 to this repo?
Ping on this, @bgruening, @lldelisle
@isuruf: @bgruening, @lldelisle, can you send a PR to port xianyi/OpenBLAS#2729 to this repo?
Done in #103.
Issue:
We got a coredump with the latest version 0.3.10 (build 2): https://travis-ci.com/github/lldelisle/pyGenomeTracks/jobs/362129644#L974
libopenblas-0.3.9
seems to work for us. Both versions (also 0.3.10-openmp_h709eae2_2) seems to cause the segfault.Environment (
conda list
):Details about
conda
and system (conda info
):