Using NUMA requires -lnuma to be passed to GHC

[izgzhen]

collect2: error: ld returned 1 exit status
`gcc' failed in phase `Linker'. (Exit code: 1)
shakeArgsWith    0.000s    0%                           
Function shake  12.998s    4%  =                        
Database read    0.251s    0%                           
With database    0.012s    0%                           
Running rules  267.347s   95%  =========================
Total          280.609s  100%                           
Error when running Shake build system:
* _build/stage1/bin/runhaskell
user error (Development.Shake.cmd, system command failed
Command: _build/stage0/bin/ghc -Wall -hisuf hi -osuf o -hcsuf hc -static -hide-all-packages -no-user-package-db '-package-db _build/stage1/lib/package.conf.d' '-package-id base-4.11.0.0' '-package-id directory-1.3.1.5' '-package-id filepath-1.4.2' '-package-id process-1.6.3.0' '-package-id unix-2.7.2.2' -i -i_build/stage1/utils/runghc/build -i_build/stage1/utils/runghc/build/runghc/autogen -iutils/runghc/. -Iincludes -I_build/generated -I_build/stage1/utils/runghc/build -I/home/zhen/repos/ghc/_build/stage1/lib/x86_64-linux-ghc-8.5.20180406/process-1.6.3.0/include -I/home/zhen/repos/ghc/_build/stage1/lib/x86_64-linux-ghc-8.5.20180406/unix-2.7.2.2/include -I/home/zhen/repos/ghc/_build/stage1/lib/x86_64-linux-ghc-8.5.20180406/time-1.8.0.2/include -I/home/zhen/repos/ghc/_build/stage1/lib/x86_64-linux-ghc-8.5.20180406/bytestring-0.10.8.2/include -I/home/zhen/repos/ghc/_build/stage1/lib/x86_64-linux-ghc-8.5.20180406/base-4.11.0.0/include -I/home/zhen/repos/ghc/_build/stage1/lib/x86_64-linux-ghc-8.5.20180406/rts-1.0/include -I_build/generated -optc-I_build/generated -optP-include -optP_build/stage1/utils/runghc/build/runghc/autogen/cabal_macros.h -optc-fno-stack-protector -odir _build/stage1/utils/runghc/build -hidir _build/stage1/utils/runghc/build -stubdir _build/stage1/utils/runghc/build -no-auto-link-packages -rtsopts -Wnoncanonical-monad-instances -optc-Werror=unused-but-set-variable -optc-Wno-error=inline _build/stage1/utils/runghc/build/Main.o -o _build/stage1/bin/runhaskell -O0 -H64m -XHaskell2010 -ghcversion-file=/home/zhen/repos/ghc/_build/generated/ghcversion.h
Exit code: 1
Stderr:
_build/stage1/lib/../lib/x86_64-linux-ghc-8.5.20180406/rts-1.0/libHSrts-1.0.a(OSMem.o):function osBindMBlocksToNode: error: undefined reference to 'mbind'
_build/stage1/lib/../lib/x86_64-linux-ghc-8.5.20180406/rts-1.0/libHSrts-1.0.a(OSMem.o):function osNumaAvailable: error: undefined reference to 'numa_available'
_build/stage1/lib/../lib/x86_64-linux-ghc-8.5.20180406/rts-1.0/libHSrts-1.0.a(OSMem.o):function osNumaMask: error: undefined reference to 'numa_get_mems_allowed'
_build/stage1/lib/../lib/x86_64-linux-ghc-8.5.20180406/rts-1.0/libHSrts-1.0.a(OSMem.o):function osNumaMask: error: undefined reference to 'numa_num_configured_nodes'
_build/stage1/lib/../lib/x86_64-linux-ghc-8.5.20180406/rts-1.0/libHSrts-1.0.a(OSMem.o):function osNumaMask: error: undefined reference to 'numa_bitmask_free'
_build/stage1/lib/../lib/x86_64-linux-ghc-8.5.20180406/rts-1.0/libHSrts-1.0.a(OSMem.o):function osNumaMask: error: undefined reference to 'numa_num_configured_nodes'
_build/stage1/lib/../lib/x86_64-linux-ghc-8.5.20180406/rts-1.0/libHSrts-1.0.a(OSMem.o):function osNumaNodes: error: undefined reference to 'numa_num_configured_nodes'
_build/stage1/lib/../lib/x86_64-linux-ghc-8.5.20180406/rts-1.0/libHSrts-1.0.a(OSThreads.o):function setThreadNode: error: undefined reference to 'numa_run_on_node'
_build/stage1/lib/../lib/x86_64-linux-ghc-8.5.20180406/rts-1.0/libHSrts-1.0.a(OSThreads.o):function releaseThreadNode: error: undefined reference to 'numa_run_on_node'
collect2: error: ld returned 1 exit status
`gcc' failed in phase `Linker'. (Exit code: 1)
)

[izgzhen]

adding -lnuma to ghc will fix

[snowleopard]

@izgzhen Thanks! I didn't find any -lnuma flag in the Make build system though.

[izgzhen]

@snowleopard true, and I don't know why Make doesn't need this...

[izgzhen]

and ./build.sh --flavour=quickest --integer-simple -j can finally work with the HACKs in #554 and this issue.

[snowleopard]

@izgzhen We now have green CircleCI. Is this still an issue on your machine?

[izgzhen]

Emm, i guess it is because I built it on a DELL server:

Architecture:          x86_64
CPU op-mode(s):        32-bit, 64-bit
Byte Order:            Little Endian
CPU(s):                40
On-line CPU(s) list:   0-39
Thread(s) per core:    2
Core(s) per socket:    10
Socket(s):             2
NUMA node(s):          2
Vendor ID:             GenuineIntel
CPU family:            6
Model:                 79
Model name:            Intel(R) Xeon(R) CPU E5-2630 v4 @ 2.20GHz
Stepping:              1
CPU MHz:               1221.257
CPU max MHz:           3100.0000
CPU min MHz:           1200.0000
BogoMIPS:              4401.41
Virtualization:        VT-x
L1d cache:             32K
L1i cache:             32K
L2 cache:              256K
L3 cache:              25600K
NUMA node0 CPU(s):     0,2,4,6,8,10,12,14,16,18,20,22,24,26,28,30,32,34,36,38
NUMA node1 CPU(s):     1,3,5,7,9,11,13,15,17,19,21,23,25,27,29,31,33,35,37,39
Flags:                 fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm 3dnowprefetch epb invpcid_single intel_pt retpoline kaiser tpr_shadow vnmi flexpriority ept vpid fsgsbase tsc_adjust bmi1 hle avx2 smep bmi2 erms invpcid rtm cqm rdseed adx smap xsaveopt cqm_llc cqm_occup_llc cqm_mbm_total cq

In this environment, the error will happen.

[snowleopard]

@izgzhen I have a suspicion that the Make build system has the same issue.

Perhaps, this is just a matter of updating the documentation and not just in Hadrian, but in GHC as well, by saying that one needs to add -lnuma when building on certain architectures.

[izgzhen]

@izgzhen I have a suspicion that the Make build system has the same issue.

It built fine with Make...

[snowleopard]

Oops, that's unexpected. I wonder what's different in Hadrian...

[bgamari]

I think https://phabricator.haskell.org/D4616 should fix this.

[snowleopard]

@bgamari Awesome! Thank you.

[bgamari]

D4617 is another approach which has the additional benefit of eliminating the need to produce both NUMA-compatible and vanilla bindists.

[izgzhen]

How is this going on now?

[bgamari]

I finished off D4617 today. I'll likely merge it tomorrow.

[bgamari]

For the record, D4616 has been merged to master (as 45ad0c3946623bc8635d6e3964d9ebb1555a3df6) so this ought to work now.

We might also adopt D4617, but this remains to be seen.