search

ClangBuiltLinux / linux

Linux kernel source tree
Other
242 stars 14 forks source link

lld: x86_64: sysfs: cannot create duplicate filename $module/.rodata.cst{16,32} #431

Closed dileks closed 5 years ago

dileks commented 5 years ago

In dmesg I am seeing:

$ grep sysfs: dmesg_5.0.4-rc1-1-amd64-cbl-asmgoto.txt 
[Fri Mar 22 10:32:09 2019] sysfs: cannot create duplicate filename '/module/usbcore/sections/.rodata.cst16'
[Fri Mar 22 10:32:18 2019] sysfs: cannot create duplicate filename '/module/nfsd/sections/.rodata.cst32'
[Fri Mar 22 10:32:18 2019] sysfs: cannot create duplicate filename '/module/iwlwifi/sections/.rodata.cst16'
[Fri Mar 22 10:32:18 2019] sysfs: cannot create duplicate filename '/module/i915/sections/.rodata.cst32'
[Fri Mar 22 10:32:18 2019] sysfs: cannot create duplicate filename '/module/mac80211/sections/.rodata.cst32'
[Fri Mar 22 10:32:18 2019] sysfs: cannot create duplicate filename '/module/iwlmvm/sections/.rodata.cst16'
[Fri Mar 22 10:32:20 2019] sysfs: cannot create duplicate filename '/module/bluetooth/sections/.rodata.cst16'

This is mostly due to .rodata.cst{16,32} sections in:

arch/x86/crypto/*-asm.S

As an example:

$ git --no-pager grep -E 'rodata.cst32|rodata.cst16' | grep sha512
arch/x86/crypto/sha512-avx-asm.S:.section       .rodata.cst16.XMM_QWORD_BSWAP, "aM", @progbits, 16
arch/x86/crypto/sha512-avx2-asm.S:.section      .rodata.cst32.PSHUFFLE_BYTE_FLIP_MASK, "aM", @progbits, 32
arch/x86/crypto/sha512-avx2-asm.S:.section      .rodata.cst32.MASK_YMM_LO, "aM", @progbits, 32
arch/x86/crypto/sha512-ssse3-asm.S:.section     .rodata.cst16.XMM_QWORD_BSWAP, "aM", @progbits, 16

Not sure if above modules select/depend on x86/crypto stuff (I just looked at mac80211). I have not looked deeper into this.

nickdesaulniers commented 5 years ago

cc @rui314 @GeorgiiR @smithp35

smithp35 commented 5 years ago

Can I get a bit more context about which file sysfs is complaining about, and whether it has been produced by ld -r or not?

AFAIK the .rodata.cst are merge sections which a linker is permitted, but not required, to merge as an optimisation. I'm guessing that LLD hasn't done the merge (possibly because of ld -r) and the names have been truncated on output leading to duplicate section names. I guessing sysfs can't cope with this?

dileks commented 5 years ago

Sorry, was busy at work.

Attached are my dmesg-log and kernel-config files.

Excerpt for mac80211:

[Fri Mar 22 10:32:18 2019] sysfs: cannot create duplicate filename '/module/mac80211/sections/.rodata.cst32'
[Fri Mar 22 10:32:18 2019] CPU: 0 PID: 482 Comm: modprobe Not tainted 5.0.4-rc1-1-amd64-cbl-asmgoto #1~buster+dileks1
[Fri Mar 22 10:32:18 2019] Hardware name: LENOVO 20HDCTO1WW/20HDCTO1WW, BIOS N1QET81W (1.56 ) 02/20/2019
[Fri Mar 22 10:32:18 2019] Call Trace:
[Fri Mar 22 10:32:18 2019]  dump_stack+0x54/0x8b
[Fri Mar 22 10:32:18 2019]  sysfs_warn_dup+0x5e/0x70
[Fri Mar 22 10:32:18 2019]  sysfs_add_file_mode_ns+0x130/0x160
[Fri Mar 22 10:32:18 2019]  internal_create_group+0x1f6/0x440
[Fri Mar 22 10:32:18 2019]  load_module+0x2f41/0x31b0
[Fri Mar 22 10:32:18 2019]  ? kernel_read_file_from_fd+0x46/0x80
[Fri Mar 22 10:32:18 2019]  __do_sys_finit_module+0x150/0x180
[Fri Mar 22 10:32:18 2019]  do_syscall_64+0x56/0x70
[Fri Mar 22 10:32:18 2019]  entry_SYSCALL_64_after_hwframe+0x44/0xa9
[Fri Mar 22 10:32:18 2019] RIP: 0033:0x7f43449ed2a9
[Fri Mar 22 10:32:18 2019] Code: 00 c3 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 44 00 00 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d b7 6b 0c 00 f7 d8 64 89 01 48
[Fri Mar 22 10:32:18 2019] RSP: 002b:00007ffd48bc0f18 EFLAGS: 00000246 ORIG_RAX: 0000000000000139
[Fri Mar 22 10:32:18 2019] RAX: ffffffffffffffda RBX: 0000562d096f40c0 RCX: 00007f43449ed2a9
[Fri Mar 22 10:32:18 2019] RDX: 0000000000000000 RSI: 0000562d092eb3f0 RDI: 0000000000000000
[Fri Mar 22 10:32:18 2019] RBP: 0000562d092eb3f0 R08: 0000000000000000 R09: 0000000000000000
[Fri Mar 22 10:32:18 2019] R10: 0000000000000000 R11: 0000000000000246 R12: 0000000000000000
[Fri Mar 22 10:32:18 2019] R13: 0000562d096f4060 R14: 0000000000040000 R15: 0000562d096f40c0

Not sure if there is an interaction with systemd-udevd (here bluetooth module):

[Fri Mar 22 10:32:20 2019] sysfs: cannot create duplicate filename '/module/bluetooth/sections/.rodata.cst16'
[Fri Mar 22 10:32:20 2019] CPU: 3 PID: 430 Comm: systemd-udevd Not tainted 5.0.4-rc1-1-amd64-cbl-asmgoto #1~buster+dileks1
[Fri Mar 22 10:32:20 2019] Hardware name: LENOVO 20HDCTO1WW/20HDCTO1WW, BIOS N1QET81W (1.56 ) 02/20/2019
[Fri Mar 22 10:32:20 2019] Call Trace:
[Fri Mar 22 10:32:20 2019]  dump_stack+0x54/0x8b
[Fri Mar 22 10:32:20 2019]  sysfs_warn_dup+0x5e/0x70
[Fri Mar 22 10:32:20 2019]  sysfs_add_file_mode_ns+0x130/0x160
[Fri Mar 22 10:32:20 2019]  internal_create_group+0x1f6/0x440
[Fri Mar 22 10:32:20 2019]  load_module+0x2f41/0x31b0
[Fri Mar 22 10:32:20 2019]  ? kernel_read_file_from_fd+0x46/0x80
[Fri Mar 22 10:32:20 2019]  __do_sys_finit_module+0x150/0x180
[Fri Mar 22 10:32:20 2019]  do_syscall_64+0x56/0x70
[Fri Mar 22 10:32:20 2019]  entry_SYSCALL_64_after_hwframe+0x44/0xa9
[Fri Mar 22 10:32:20 2019] RIP: 0033:0x7f0b899492a9
[Fri Mar 22 10:32:20 2019] Code: 00 c3 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 44 00 00 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d b7 6b 0c 00 f7 d8 64 89 01 48
[Fri Mar 22 10:32:20 2019] RSP: 002b:00007ffc04020268 EFLAGS: 00000246 ORIG_RAX: 0000000000000139
[Fri Mar 22 10:32:20 2019] RAX: ffffffffffffffda RBX: 0000559c7e3c46e0 RCX: 00007f0b899492a9
[Fri Mar 22 10:32:20 2019] RDX: 0000000000000000 RSI: 00007f0b8984dcad RDI: 000000000000000f
[Fri Mar 22 10:32:20 2019] RBP: 00007f0b8984dcad R08: 0000000000000000 R09: 0000000000000000
[Fri Mar 22 10:32:20 2019] R10: 000000000000000f R11: 0000000000000246 R12: 0000000000000000
[Fri Mar 22 10:32:20 2019] R13: 0000559c7e3cf0a0 R14: 0000000000020000 R15: 0000559c7e3c46e0

dmesg_5.0.4-rc1-1-amd64-cbl-asmgoto_anonymized.txt config-5.0.4-rc1-1-amd64-cbl-asmgoto.txt

smithp35 commented 5 years ago

Thanks. I've done a quick check on how LLD names orphan merge sections (orphan means not covered by linker script). When linking a full executable or shared library. For an input section name .rodata. LLD will place it in an OutputSection called .rodata When linking a relocatable (-r) object file LLD does not do any merging and preserves the input section name.

So we I think we know that the -r option is being used, LLD isn't silently truncating the section names and it is not merging sections when the -r option is on.

From the error message: [Fri Mar 22 10:32:09 2019] sysfs: cannot create duplicate filename '/module/usbcore/sections/.rodata.cst16' [Fri Mar 22 10:32:18 2019] sysfs: cannot create duplicate filename '/module/nfsd/sections/.rodata.cst32'

I found at least one .section .rodata.cst16, "aM", @progbits, 16 in the kernel but I didn't find any .section .rodata.cst32, "aM", @progbits, 32 This would imply that at least the .rodata.cst32 sections came from the compiler, which looking at the tests suggest it can generate.

May be worth compiling with -fdata-sections to see if the .rodata.cst sections gain a suffix.

dileks commented 5 years ago

Looks like I should build with CONFIG_LD_DEAD_CODE_DATA_ELIMINATION=y (see [1]).

...defined in init/Kconfig and with init/Makefile including ccflags-y := -fno-function-sections -fno-data-sections...

Checked this one:

[ include/asm-generic/vmlinux.lds.h ]

/*
 * LD_DEAD_CODE_DATA_ELIMINATION option enables -fdata-sections, which
 * generates .data.identifier sections, which need to be pulled in with
 * .data. We don't want to pull in .data..other sections, which Linux
 * has defined. Same for text and bss.
 *
 * RODATA_MAIN is not used because existing code already defines .rodata.x
 * sections to be brought in with rodata.
 */
#ifdef CONFIG_LD_DEAD_CODE_DATA_ELIMINATION
#define TEXT_MAIN .text .text.[0-9a-zA-Z_]*
#define DATA_MAIN .data .data.[0-9a-zA-Z_]* .data..LPBX*
#define SDATA_MAIN .sdata .sdata.[0-9a-zA-Z_]*
#define RODATA_MAIN .rodata .rodata.[0-9a-zA-Z_]* <--- XXX: Look here!
#define BSS_MAIN .bss .bss.[0-9a-zA-Z_]*
#define SBSS_MAIN .sbss .sbss.[0-9a-zA-Z_]*
#else
#define TEXT_MAIN .text
#define DATA_MAIN .data
#define SDATA_MAIN .sdata
#define RODATA_MAIN .rodata
#define BSS_MAIN .bss
#define SBSS_MAIN .sbss
#endif

[1] https://cateee.net/lkddb/web-lkddb/LD_DEAD_CODE_DATA_ELIMINATION.html

dileks commented 5 years ago

x86 arch has no HAVE_LD_DEAD_CODE_DATA_ELIMINATION:

$ git grep LD_DEAD_CODE_DATA_ELIMINATION | grep arch
arch/mips/Kconfig:      select HAVE_LD_DEAD_CODE_DATA_ELIMINATION
arch/powerpc/Kconfig:   select HAVE_LD_DEAD_CODE_DATA_ELIMINATION

From init/Kconfig:

config HAVE_LD_DEAD_CODE_DATA_ELIMINATION
        bool
        help
          This requires that the arch annotates or otherwise protects
          its external entry points from being discarded. Linker scripts
          must also merge .text.*, .data.*, and .bss.* correctly into
          output sections. Care must be taken not to pull in unrelated
          sections (e.g., '.text.init'). Typically '.' in section names
          is used to distinguish them from label names / C identifiers.

config LD_DEAD_CODE_DATA_ELIMINATION
        bool "Dead code and data elimination (EXPERIMENTAL)"
        depends on HAVE_LD_DEAD_CODE_DATA_ELIMINATION
        depends on EXPERT
        depends on !(FUNCTION_TRACER && CC_IS_GCC && GCC_VERSION < 40800)
        depends on $(cc-option,-ffunction-sections -fdata-sections)
        depends on $(ld-option,--gc-sections)
        help
          Enable this if you want to do dead code and data elimination with
          the linker by compiling with -ffunction-sections -fdata-sections,
          and linking with --gc-sections.

          This can reduce on disk and in-memory size of the kernel
          code and static data, particularly for small configs and
          on small systems. This has the possibility of introducing
          silently broken kernel if the required annotations are not
          present. This option is not well tested yet, so use at your
          own risk.

I try with -fdata-sectionsin the toplevel Makefile.

dileks commented 5 years ago

Even with the LLD patch "Fix -emit-reloc against local symbols. " and Linux-kernel patch "x86/build: Keep local relocations with ld.lld" I still see the above messages.

This seems to affect Linux-kernel modules only. Not sure if I pass --discard-none in the toplevel Makefile?

[1] https://github.com/llvm-mirror/lld/commit/ac598e868665dff6fca0f6607ed77ad72717a5ba [2] https://git.kernel.org/pub/scm/linux/kernel/git/tip/tip.git/commit/?h=x86/build&id=7c21383f3429dd70da39c0c7f1efa12377a47ab6

E5ten commented 5 years ago

I don't get those warnings but I don't have any modules in my kernel.

dileks commented 5 years ago

"GCC puts its mergeable constants in ".rodata.cstSIZE" sections, or ".rodata.cstSIZE." if -fdata-sections is used."

commit e183914af00e15eb41ae666d44e323bfa154be13 ("crypto: x86 - make constants readonly, allow linker to merge them")

[1] https://git.kernel.org/linus/e183914af00e15eb41ae666d44e323bfa154be13

dileks commented 5 years ago

I posted to LKML and x86 ML.

UPDATE: Thread "CBL issue #431: lld: x86_64: sysfs: cannot create duplicate filename $module/.rodata.cst{16,32}"

[1] https://lore.kernel.org/r/CA+icZUVW2X6G=qukJTrZuUevcnrOiuShJGxnxVaqyONNxv+RYg@mail.gmail.com/

dileks commented 5 years ago

I was able to build and link with CONFIG_LD_DEAD_CODE_DATA_ELIMINATION=y but cannot boot on bare metal (even with nokaslr boot-parameter).

I needed this snippet:

diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig
index 64d5a3327030..6df511429c28 100644
--- a/arch/x86/Kconfig
+++ b/arch/x86/Kconfig
@@ -167,6 +167,7 @@ config X86
        select HAVE_KPROBES_ON_FTRACE
        select HAVE_FUNCTION_ERROR_INJECTION
        select HAVE_KRETPROBES
+       select HAVE_LD_DEAD_CODE_DATA_ELIMINATION if X86_64
        select HAVE_KVM
        select HAVE_LIVEPATCH                   if X86_64
        select HAVE_MEMBLOCK_NODE_MAP

Investigating QEMU log:

$ cat run_qemu.sh 
KPATH=$(pwd)

APPEND="root=/dev/ram0 console=ttyS0 hung_task_panic=1 earlyprintk=ttyS0,115200"
##APPEND="$APPEND nokaslr"

qemu-system-x86_64 -enable-kvm -M pc -kernel $KPATH/bzImage -initrd $KPATH/initrd.img -m 512 -net none -serial stdio -append "${APPEND}"

$ ./run_qemu.sh   
Probing EDD (edd=off to disable)... ok
early console in extract_kernel
input_data: 0x00000000025263b1
input_len: 0x000000000049c67c
output: 0x0000000001000000
output_len: 0x0000000001998404
kernel_total_size: 0x00000000016ae000
trampoline_32bit: 0x000000000009d000
booted via startup_32()
Physical KASLR using RDTSC...
Virtual KASLR using RDTSC...

Decompressing Linux... Parsing ELF... Performing relocations... done.
Booting the kernel.
[    0.000000] Linux version 5.0.7-2-amd64-cbl-asmgoto (sedat.dilek@gmail.com@iniza) (clang version 9.0.0 (https://github.com/llvm-mirror/clang.git b4f4320de39f9a7227af0a8374c6739af6969f7a) (https://github.com/llvm-mirror/llvm.git 1fd2bab1d52f7a37c9d385b8af20b26f1737bf7c)) #2~buster+dileks1 SMP 2019-04-08
[    0.000000] Command line: root=/dev/ram0 console=ttyS0 hung_task_panic=1 earlyprintk=ttyS0,115200
[    0.000000] CPU: vendor_id 'GenuineIntel' unknown, using generic init.
[    0.000000] CPU: Your system may be unstable.
[    0.000000] x86/fpu: x87 FPU will use FXSAVE
[    0.000000] BIOS-provided physical RAM map:
[    0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable
[    0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved
[    0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved
[    0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000001ffdffff] usable
[    0.000000] BIOS-e820: [mem 0x000000001ffe0000-0x000000001fffffff] reserved
[    0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved
[    0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved
[    0.000000] printk: bootconsole [earlyser0] enabled
[    0.000000] NX (Execute Disable) protection: active
[    0.000000] SMBIOS 2.8 present.
[    0.000000] DMI: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.12.0-1 04/01/2014
[    0.000000] Hypervisor detected: KVM
[    0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00
[    0.000000] kvm-clock: cpu 0, msr 8016001, primary cpu clock
[    0.000000] kvm-clock: using sched offset of 1386508206 cycles
[    0.000590] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns
[    0.002248] tsc: Detected 2904.000 MHz processor
[    0.004183] last_pfn = 0x1ffe0 max_arch_pfn = 0x400000000
[    0.005072] x86/PAT: Configuration [0-7]: WB  WC  UC- UC  WB  WP  UC- WT  
Memory KASLR using RDTSC...
[    0.009505] found SMP MP-table at [mem 0x000f5aa0-0x000f5aaf]
[    0.010319] RAMDISK: [mem 0x1dfa8000-0x1ffdffff]
[    0.010828] ACPI: Early table checksum verification disabled
[    0.011445] ACPI: RSDP 0x00000000000F58D0 000014 (v00 BOCHS )
[    0.012042] ACPI: RSDT 0x000000001FFE156F 000030 (v01 BOCHS  BXPCRSDT 00000001 BXPC 00000001)
[    0.012933] ACPI: FACP 0x000000001FFE144B 000074 (v01 BOCHS  BXPCFACP 00000001 BXPC 00000001)
[    0.013837] ACPI: DSDT 0x000000001FFE0040 00140B (v01 BOCHS  BXPCDSDT 00000001 BXPC 00000001)
[    0.014715] ACPI: FACS 0x000000001FFE0000 000040
[    0.015182] ACPI: APIC 0x000000001FFE14BF 000078 (v01 BOCHS  BXPCAPIC 00000001 BXPC 00000001)
[    0.016042] ACPI: HPET 0x000000001FFE1537 000038 (v01 BOCHS  BXPCHPET 00000001 BXPC 00000001)
[    0.017131] No NUMA configuration found
[    0.017521] Faking a node at [mem 0x0000000000000000-0x000000001ffdffff]
[    0.018208] NODE_DATA(0) allocated [mem 0x1dfa3000-0x1dfa7fff]
[    0.018924] Zone ranges:
[    0.019187]   DMA      [mem 0x0000000000001000-0x0000000000ffffff]
[    0.019822]   DMA32    [mem 0x0000000001000000-0x000000001ffdffff]
[    0.020461]   Normal   empty
[    0.020753]   Device   empty
[    0.021039] Movable zone start for each node
[    0.021679] Early memory node ranges
[    0.022153]   node   0: [mem 0x0000000000001000-0x000000000009efff]
[    0.022788]   node   0: [mem 0x0000000000100000-0x000000001ffdffff]
[    0.023505] Zeroed struct page in unavailable ranges: 98 pages
[    0.023507] Initmem setup node 0 [mem 0x0000000000001000-0x000000001ffdffff]
[    0.026277] ACPI: PM-Timer IO Port: 0x608
[    0.026665] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])
[    0.027403] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23
[    0.028193] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)
[    0.028920] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)
[    0.029696] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)
[    0.030371] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)
[    0.031059] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)
[    0.031755] Using ACPI (MADT) for SMP configuration information
[    0.032409] ACPI: HPET id: 0x8086a201 base: 0xfed00000
[    0.032948] smpboot: Allowing 1 CPUs, 0 hotplug CPUs
[    0.033460] PM: Registered nosave memory: [mem 0x00000000-0x00000fff]
[    0.034113] PM: Registered nosave memory: [mem 0x0009f000-0x0009ffff]
[    0.034768] PM: Registered nosave memory: [mem 0x000a0000-0x000effff]
[    0.035776] PM: Registered nosave memory: [mem 0x000f0000-0x000fffff]
[    0.036565] [mem 0x20000000-0xfeffbfff] available for PCI devices
[    0.037288] Booting paravirtualized kernel on KVM
[    0.037851] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645519600211568 ns
[    0.103725] random: get_random_bytes called from start_kernel+0x93/0x466 with crng_init=0
[    0.104881] setup_percpu: NR_CPUS:512 nr_cpumask_bits:512 nr_cpu_ids:1 nr_node_ids:1
[    0.108866] percpu: Embedded 44 pages/cpu @(____ptrval____) s141784 r8192 d30248 u2097152
[    0.110412] KVM setup async PF for cpu 0
[    0.110839] kvm-stealtime: cpu 0, msr 1d216080
[    0.111409] Built 1 zonelists, mobility grouping on.  Total pages: 128872
[    0.112152] Policy zone: DMA32
[    0.112509] Kernel command line: root=/dev/ram0 console=ttyS0 hung_task_panic=1 earlyprintk=ttyS0,115200
[    0.114983] Memory: 459244K/523768K available (10243K kernel code, 1091K rwdata, 2996K rodata, 1492K init, 1220K bss, 64524K reserved, 0K cma-reserved)
[    0.116623] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1
[    0.117312] Kernel/User page tables isolation: enabled
[    0.117855] ftrace: allocating 29347 entries in 115 pages
[    0.126894] rcu: Hierarchical RCU implementation.
[    0.127524] rcu:     RCU restricting CPUs from NR_CPUS=512 to nr_cpu_ids=1.
[    0.128241] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.
[    0.129038] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1
[    0.131267] NR_IRQS: 33024, nr_irqs: 256, preallocated irqs: 16
[    0.139117] Console: colour VGA+ 80x25
[    0.139625] printk: console [ttyS0] enabled
[    0.139625] printk: console [ttyS0] enabled
[    0.140665] printk: bootconsole [earlyser0] disabled
[    0.140665] printk: bootconsole [earlyser0] disabled
[    0.141905] ACPI: Core revision 20181213
[    0.142587] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns
[    0.143806] APIC: Switch to symmetric I/O mode setup
[    0.145031] x2apic enabled
[    0.146897] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1
[    0.147689] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x29dc05e54fc, max_idle_ns: 440795291716 ns
[    0.149097] Calibrating delay loop (skipped) preset value.. 5808.00 BogoMIPS (lpj=11616000)
[    0.150126] pid_max: default: 32768 minimum: 301
[    0.150736] LSM: Security Framework initializing
[    0.151322] Yama: becoming mindful.
[    0.152038] AppArmor: AppArmor initialized
[    0.152791] Dentry cache hash table entries: 65536 (order: 7, 524288 bytes)
[    0.153087] Inode-cache hash table entries: 32768 (order: 6, 262144 bytes)
[    0.153087] Mount-cache hash table entries: 1024 (order: 1, 8192 bytes)
[    0.153087] Mountpoint-cache hash table entries: 1024 (order: 1, 8192 bytes)
[    0.153087] mce: CPU supports 10 MCE banks
[    0.153087] mce: unknown CPU type - not enabling MCE support
[    0.153087] Last level iTLB entries: 4KB 0, 2MB 0, 4MB 0
[    0.153087] Last level dTLB entries: 4KB 0, 2MB 0, 4MB 0, 1GB 0
[    0.153087] Spectre V2 : Mitigation: Full generic retpoline
[    0.153087] Spectre V2 : Spectre v2 / SpectreRSB mitigation: Filling RSB on context switch
[    0.153087] Speculative Store Bypass: Vulnerable
qemu-system-x86_64: terminating on signal 2

Any ideas?

dileks commented 5 years ago

Will look if the patch "x86/build/lto: Fix truncated .bss with -fdata-sections" from #450 helps me.

[1] https://git.kernel.org/pub/scm/linux/kernel/git/tip/tip.git/commit/?h=x86/urgent&id=6a03469a1edc94da52b65478f1e00837add869a3

dileks commented 5 years ago

With above patch and the following snippet...

diff --git a/Makefile b/Makefile
index ac29cd533d55..14e25e01a2a5 100644
--- a/Makefile
+++ b/Makefile
@@ -813,7 +813,7 @@ endif

 ifdef CONFIG_LD_DEAD_CODE_DATA_ELIMINATION
 KBUILD_CFLAGS_KERNEL += -ffunction-sections -fdata-sections
-LDFLAGS_vmlinux += --gc-sections
+#LDFLAGS_vmlinux += --gc-sections
 endif

...I can build, link and boot in QEMU.

So, there is a problem with --gc-sections when CONFIG_LD_DEAD_CODE_DATA_ELIMINATION=y.

I still see messages like...

[    1.306575] sysfs: cannot create duplicate filename '/module/libata/sections/.rodata.cst8'
dileks commented 5 years ago

As this affects Linux-kernel modules I will try with this:

diff --git a/Makefile b/Makefile
index 0c528f7b0f36..39dfbfe7a30b 100644
--- a/Makefile
+++ b/Makefile
@@ -809,8 +809,11 @@ endif

 ifdef CONFIG_LD_DEAD_CODE_DATA_ELIMINATION
 KBUILD_CFLAGS_KERNEL += -ffunction-sections -fdata-sections
+KBUILD_CFLAGS_MODULE += -ffunction-sections -fdata-sections
+ifndef CONFIG_LD_IS_LLD
 LDFLAGS_vmlinux += --gc-sections
 endif
+endif

 # arch Makefile may override CC so keep this after arch Makefile is included
 NOSTDINC_FLAGS += -nostdinc -isystem $(shell $(CC) -print-file-name=include)
diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig
index 64d5a3327030..6df511429c28 100644
--- a/arch/x86/Kconfig
+++ b/arch/x86/Kconfig
@@ -167,6 +167,7 @@ config X86
        select HAVE_KPROBES_ON_FTRACE
        select HAVE_FUNCTION_ERROR_INJECTION
        select HAVE_KRETPROBES
+       select HAVE_LD_DEAD_CODE_DATA_ELIMINATION if X86_64
        select HAVE_KVM
        select HAVE_LIVEPATCH                   if X86_64
        select HAVE_MEMBLOCK_NODE_MAP
dileks commented 5 years ago

I found these two commits:

parisc: Build kernel without -ffunction-sections
parisc: Enable -ffunction-sections for modules on 32-bit kernel

And [2] says:

Frank Schreiner reported, that since kernel 4.18 he faces sysfs-warnings
when loading modules on a 32-bit kernel. Here is one such example:

 sysfs: cannot create duplicate filename '/module/nfs/sections/.text'
 CPU: 0 PID: 98 Comm: modprobe Not tainted 4.18.0-2-parisc #1 Debian 4.18.10-2
 Backtrace:
  [<1017ce2c>] show_stack+0x3c/0x50
  [<107a7210>] dump_stack+0x28/0x38
  [<103f900c>] sysfs_warn_dup+0x88/0xac
  [<103f8b1c>] sysfs_add_file_mode_ns+0x164/0x1d0
  [<103f9e70>] internal_create_group+0x11c/0x304
  [<103fa0a0>] sysfs_create_group+0x48/0x60
  [<1022abe8>] load_module.constprop.35+0x1f9c/0x23b8
  [<1022b278>] sys_finit_module+0xd0/0x11c
  [<101831dc>] syscall_exit+0x0/0x14

This warning gets triggered by the fact, that due to commit 24b6c22504a2
("parisc: Build kernel without -ffunction-sections") we now get multiple .text
sections in the kernel modules for which sysfs_create_group() can't create
multiple virtual files.

This patch works around the problem by re-enabling the -ffunction-sections
compiler option for modules, while keeping it disabled for the non-module
kernel code.

[3] says (kbuild: LD_DEAD_CODE_DATA_ELIMINATION no -ffunction-sections/-fdata-sections for module build):

Modules do not tend to cope with -ffunction-sections, even though they
do not link with -gc-sections. It may be possible for unused symbols to
be trimmed from modules, but in general that would take much more work
in architecture module linker scripts.

For now, enable these only for kernel build.

[1] https://git.kernel.org/linus/24b6c22504a27210a8377e54d24d425ae414f2c1 [2] https://git.kernel.org/linus/1e8249b8a4e960018e4baca6b523b8a4500af600 [3] https://git.kernel.org/linus/6ca8d9433d7712d4c3c6102816b685470504dc3d

dileks commented 5 years ago

cc @rui314 @GeorgiiR @smithp35

For further tests, I switched to GCC-8 as compiler and kept my LLD.

This sets in the toplevel-Makefile -fno-merge-all-constants and -fmerge-constants. The 2nd parameter is not available for clang.

This indeed adds a suffix to rodata.cst* sections like here:

$ grep rodata.cst32 objdump-D-Mintel_sha512-avx2-asm_o_gcc-8.txt
Disassembly of section .rodata.cst32.PSHUFFLE_BYTE_FLIP_MASK:
Disassembly of section .rodata.cst32.MASK_YMM_LO:

The affected kernel-modules like nfsd.ko has none of a rodata.cst32 section! To recall I saw this:

[Thu Apr 18 21:59:17 2019] sysfs: cannot create duplicate filename '/module/nfsd/sections/.rodata.cst32'

[ CLANG TRYOUTS ]

When using CLANG -mno-global-merge and -fno-merge-all-constants is set.

See also Makefile:

# CLANG uses a _MergedGlobals as optimization, but this breaks modpost, as the
# source of a reference will be _MergedGlobals and not on of the whitelisted names.
# See modpost pattern 2
KBUILD_CFLAGS += $(call cc-option, -mno-global-merge,)
KBUILD_CFLAGS += $(call cc-option, -fcatch-undefined-behavior)

# clang sets -fmerge-all-constants by default as optimization, but this
# is non-conforming behavior for C and in fact breaks the kernel, so we
# need to disable it here generally.
KBUILD_CFLAGS   += $(call cc-option,-fno-merge-all-constants)

# for gcc -fno-merge-all-constants disables everything, but it is fine
# to have actual conforming behavior enabled.
KBUILD_CFLAGS   += $(call cc-option,-fmerge-constants)

A suffix is added to the rodata.cst* sections when I add -fmerge-all-constants to arch/x86/crypto/Makefile.

ccflags-y =+ -fmerge-all-constants

This does not solve the problem that affected kernel-modules get more than one rodata.cst* section. I still see the sysfs warnings.

Unsure, if I shall test with -fmerge-all-constants in the toplevel Makefile. Together with -mglobal-merge?

[ BROKEN WITH GCC-8 ]

Unfortunately, the compilation breaks with GCC-8 here:

  mylinker -m elf_x86_64  -z max-page-size=0x200000 -O2   -m elf_i386 -T arch/x86/boot/setup.ld arch/x86/boot/a20.o arch/x86/boot/bioscall.o arch/x86/boot/cmdline.o arch/x86/boot/copy.o arch/x86/boot/cpu.o arch/x86/boot/cpuflags.o arch/x86/boot/cpucheck.o arch/x86/boot/early_serial_console.o arch/x86/boot/edd.o arch/x86/boot/header.o arch/x86/boot/main.o arch/x86/boot/memory.o arch/x86/boot/pm.o arch/x86/boot/pmjump.o arch/x86/boot/printf.o arch/x86/boot/regs.o arch/x86/boot/string.o arch/x86/boot/tty.o arch/x86/boot/video.o arch/x86/boot/video-mode.o arch/x86/boot/version.o arch/x86/boot/video-vga.o arch/x86/boot/video-vesa.o arch/x86/boot/video-bios.o -o arch/x86/boot/setup.elf
ld.lld: error: init sections too big!

Increasing the corresponding ASSERT...

$ git diff
diff --git a/arch/x86/boot/setup.ld b/arch/x86/boot/setup.ld
index 0149e41d42c2..9400b8eef052 100644
--- a/arch/x86/boot/setup.ld
+++ b/arch/x86/boot/setup.ld
@@ -59,6 +59,6 @@ SECTIONS
        . = ASSERT(_end <= 0x8000, "Setup too big!");
        . = ASSERT(hdr == 0x1f1, "The setup header has the wrong offset!");
        /* Necessary for the very-old-loader check to work... */
-       . = ASSERT(__end_init <= 5*512, "init sections too big!");
+       . = ASSERT(__end_init <= 8*512, "init sections too big!");

 }

...leads to this error:

make -f ./scripts/Makefile.modpost
make -f ./scripts/Makefile.build obj=arch/x86/boot/compressed arch/x86/boot/compressed/vmlinux
  mylinker -m elf_x86_64  -z max-page-size=0x200000 -O2   -m elf_i386 -T arch/x86/boot/setup.ld arch/x86/boot/a20.o arch/x86/boot/bioscall.o arch/x86/boot/cmdline.o arch/x86/boot/copy.o arch/x86/boot/cpu.o arch/x86/boot/cpuflags.o arch/x86/boot/cpucheck.o arch/x86/boot/early_serial_console.o arch/x86/boot/edd.o arch/x86/boot/header.o arch/x86/boot/main.o arch/x86/boot/memory.o arch/x86/boot/pm.o arch/x86/boot/pmjump.o arch/x86/boot/printf.o arch/x86/boot/regs.o arch/x86/boot/string.o arch/x86/boot/tty.o arch/x86/boot/video.o arch/x86/boot/video-mode.o arch/x86/boot/version.o arch/x86/boot/video-vga.o arch/x86/boot/video-vesa.o arch/x86/boot/video-bios.o -o arch/x86/boot/setup.elf
ld.lld: error: arch/x86/boot/header.o:(.header+0x12): relocation R_386_PC8 out of range: 2598 is not in [-128, 127]
make[4]: *** [arch/x86/boot/Makefile:105: arch/x86/boot/setup.elf] Error 1
make[3]: *** [arch/x86/Makefile:284: bzImage] Error 2
dileks commented 5 years ago

I was able to build, link and boot with -fmerge-all-constants and -mglobal-merge:

sdi@iniza:~/src/linux-kernel/archives/5.0.8-5-amd64-cbl-asmgoto$ egrep 'fmerge-all-constants' build-log_5.0.8-5-amd64-cbl-asmgoto.txt | wc -l
25704
sdi@iniza:~/src/linux-kernel/archives/5.0.8-5-amd64-cbl-asmgoto$ egrep 'mglobal-merge' build-log_5.0.8-5-amd64-cbl-asmgoto.txt | wc -l
25704

I still see the sysfs warnings.

dileks commented 5 years ago

"[X86] Create mergeable constant pool entries for AVX

We supported creating mergeable constant pool entries for smaller constants but not for 32-byte AVX constants.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@261584 91177308-0d34-0410-b5e6-96231b3b80d8"

Adaptation for the case of AVX2?

[1] https://github.com/llvm-mirror/llvm/commit/b23b0fffbc838c7f8c25e3c11ed3dd7d855b1bac

nickdesaulniers commented 5 years ago

Hello from my first x86 LLD linked linux kernel. I too observe this:

$ dmesg | grep sysfs:        
[    1.969125] sysfs: cannot create duplicate filename '/module/i915/sections/.rodata.cst16'
[    3.954645] sysfs: cannot create duplicate filename '/module/bluetooth/sections/.rodata.cst16'
[    4.009524] sysfs: cannot create duplicate filename '/module/mac80211/sections/.rodata.cst32'
[    4.122455] sysfs: cannot create duplicate filename '/module/ath10k_core/sections/.rodata.cst32'

This is mostly due to .rodata.cst{16,32} sections in:

Indeed, I only see references to such sections from arch/x86/crypto/*.S files.

Looking at just the linked .ko files between ld.lld and ld.bfd; it seems that ld.lld did not merge the adjacent sections.

# ld.lld
$ readelf -S drivers/net/wireless/ath/ath10k/ath10k_core.ko | grep rodata\.cst32
  [62] .rodata.cst32     PROGBITS         0000000000000000  00a20390
  [63] .rodata.cst32     PROGBITS         0000000000000000  00a203b0

# ld.bfd
$ readelf -S drivers/net/wireless/ath/ath10k/ath10k_core.ko | grep rodata\.cst32
  [32] .rodata.cst32     PROGBITS         0000000000000000  00062f00

This would imply that at least the .rodata.cst32 sections came from the compiler, which looking at the tests suggest it can generate.

Maybe, but even so ld.bfd did a merge; ld.lld did not.

$ ld.lld -r -m elf_x86_64  -z max-page-size=0x200000 -T ./scripts/module-common.lds  --build-id  -o drivers/net/wireless/ath/ath10k/ath10k_core.ko drivers/net/wireless/ath/ath10k/ath10k_core.o drivers/net/wireless/ath/ath10k/ath10k_core.mod.o

$ ld.bfd -r -m elf_x86_64  -z max-page-size=0x200000 -T ./scripts/module-common.lds  --build-id  -o drivers/net/wireless/ath/ath10k/ath10k_core.ko drivers/net/wireless/ath/ath10k/ath10k_core.o drivers/net/wireless/ath/ath10k/ath10k_core.mod.o

The below patch forces ld.lld to merge sections (via the readelf command above). I have not boot tested it yet, nor checked that it creates sections when it should not, nor checked that the section reordering causes any issues, or alignment issues.

diff --git a/scripts/module-common.lds b/scripts/module-common.lds
index d61b9e8678e8..6a35fd282822 100644
--- a/scripts/module-common.lds
+++ b/scripts/module-common.lds
@@ -23,4 +23,7 @@ SECTIONS {
        .init_array             0 : ALIGN(8) { *(SORT(.init_array.*)) *(.init_array) }

        __jump_table            0 : ALIGN(8) { KEEP(*(__jump_table)) }
+
+       .rodata.cst16           0 : { *(.rodata.cst16) }
+       .rodata.cst32           0 : { *(.rodata.cst32) }
 }

@smithp35 , is there any way to ask LLD nicely to merge these sections like bfd does?

nickdesaulniers commented 5 years ago

Filed: https://llvm.org/pr42289

dileks commented 5 years ago

In April Nicholas Piggin suggested me:

Yeah realised that after sending the last mail, that means changing the vmlinux linker script won't help.

x86 does not seem to use a module linker script, you may have to add one perhaps -- check other archs for what they do (module.lds).

$ find ./ -name module.lds
./arch/arm/kernel/module.lds
./arch/m68k/kernel/module.lds
./arch/arm64/kernel/module.lds
./arch/powerpc/kernel/module.lds
./arch/riscv/kernel/module.lds
./arch/ia64/module.lds

Maybe that is less invasive to have this in an arch/x86/kernel/module.lds file? Not sure how to translate your snippet.

Here as an example:

$ cat arch/arm64/kernel/module.lds
SECTIONS {
        .plt (NOLOAD) : { BYTE(0) }
        .init.plt (NOLOAD) : { BYTE(0) }
        .text.ftrace_trampoline (NOLOAD) : { BYTE(0) }
}

As a sidenote: Lots of these module.lds file need a SPDX-License-Identifier.

dileks commented 5 years ago

Last but not least, do we still need these two KBUILD_CFLAGS...

# CLANG uses a _MergedGlobals as optimization, but this breaks modpost, as the
# source of a reference will be _MergedGlobals and not on of the whitelisted names.
# See modpost pattern 2
KBUILD_CFLAGS += $(call cc-option, -mno-global-merge,)
...
# clang sets -fmerge-all-constants by default as optimization, but this
# is non-conforming behavior for C and in fact breaks the kernel, so we
# need to disable it here generally.
KBUILD_CFLAGS   += $(call cc-option,-fno-merge-all-constants)

Needs some analysis I know. AFAICS I tried two months ago with clang-9 (asm-goto support) and lld-9 - I was able to build, link and boot on x86-64 bare metal.

dileks commented 5 years ago

I have applied your snippet and removed all ath10k (kernel) object files, rebuilt and looks good to me:

$ cd drivers/net/wireless/ath/ath10k
$ rm -v *.o *.ko
$ readelf -S ath10k_core.ko | grep rodata\.cst16
  [ 3] .rodata.cst16     PROGBITS         0000000000000000  00000280
$ readelf -S ath10k_core.ko | grep rodata\.cst32
  [ 4] .rodata.cst32     PROGBITS         0000000000000000  000002b0

Desired output from [1]:

$ readelf -WS ath10k_core.ko
There are 55 section headers, starting at offset 0x90eed8:

Section Headers:
  [Nr] Name              Type            Address          Off    Size   ES Flg Lk Inf Al
  [ 0]                   NULL            0000000000000000 000000 000000 00      0   0  0
  [ 1] __ksymtab         PROGBITS        0000000000000000 000040 000178 00   A  0   0  8
  [ 2] __kcrctab         PROGBITS        0000000000000000 0001b8 0000bc 00   A  0   0  1
  [ 3] .rodata.cst16     PROGBITS        0000000000000000 000280 000030 10  AM  0   0 16
  [ 4] .rodata.cst32     PROGBITS        0000000000000000 0002b0 000040 20  AM  0   0  8
  [ 5] .text             PROGBITS        0000000000000000 0002f0 038203 00  AX  0   0 16
  [ 6] .rela.text        RELA            0000000000000000 0384f8 020010 18   I 52   5  8
  [ 7] .data             PROGBITS        0000000000000000 058510 002298 00  WA  0   0 16
  [ 8] .rela.data        RELA            0000000000000000 05a7a8 000270 18   I 52   7  8
  [ 9] .bss              NOBITS          0000000000000000 05aa18 000014 00  WA  0   0  4
  [10] .rela___kcrctab+ath10k_mac_tx_push_pending RELA            0000000000000000 05aa18 000468 18   I 52   2  8
  [11] .rela___ksymtab+ath10k_mac_tx_push_pending RELA            0000000000000000 05ae80 0008d0 18   I 52   1  8
  [12] __bug_table       PROGBITS        0000000000000000 05b750 000444 00  WA  0   0  1
  [13] .rela__bug_table  RELA            0000000000000000 05bb98 001110 18   I 52  12  8
  [14] .smp_locks        PROGBITS        0000000000000000 05cca8 000054 00   A  0   0  4
  [15] .rela.smp_locks   RELA            0000000000000000 05cd00 0001f8 18   I 52  14  8
  [16] .altinstructions  PROGBITS        0000000000000000 05cef8 00004e 00   A  0   0  1
  [17] .rela.altinstructions RELA            0000000000000000 05cf48 000120 18   I 52  16  8
  [18] .altinstr_replacement PROGBITS        0000000000000000 05d068 000018 00  AX  0   0  1
  [19] .rodata.str1.1    PROGBITS        0000000000000000 05d080 006eba 01 AMS  0   0  1
  [20] __ksymtab_strings PROGBITS        0000000000000000 063f3a 0004a3 00   A  0   0  1
  [21] ".discard.addressable" PROGBITS        0000000000000000 0643e0 000178 00  WA  0   0  8
  [22] .rela".discard.addressable" RELA            0000000000000000 064558 000468 18   I 52  21  8
  [23] .rodata           PROGBITS        0000000000000000 0649c0 005998 00   A  0   0 16
  [24] .rela.rodata      RELA            0000000000000000 06a358 0041b8 18   I 52  23  8
  [25] .rodata.str4.4    PROGBITS        0000000000000000 06e510 000010 04 AMS  0   0  4
  [26] .debug_str        PROGBITS        0000000000000000 06e520 02e039 01  MS  0   0  1
  [27] .debug_loc        PROGBITS        0000000000000000 09c559 07d89b 00      0   0  1
  [28] .debug_abbrev     PROGBITS        0000000000000000 119df4 0058d2 00      0   0  1
  [29] .debug_info       PROGBITS        0000000000000000 11f6c6 2c855b 00      0   0  1
  [30] .rela.debug_info  RELA            0000000000000000 3e7c28 4ae0e0 18   I 52  29  8
  [31] .debug_ranges     PROGBITS        0000000000000000 895d08 0088d0 00      0   0  1
  [32] .debug_macinfo    PROGBITS        0000000000000000 89e5d8 000013 00      0   0  1
  [33] .comment          PROGBITS        0000000000000000 89e5eb 00004e 01  MS  0   0  1
  [34] .debug_line       PROGBITS        0000000000000000 89e639 02ff89 00      0   0  1
  [35] .rela.debug_line  RELA            0000000000000000 8ce5c8 0001b0 18   I 52  34  8
  [36] .debug_frame      PROGBITS        0000000000000000 8ce778 007158 00      0   0  8
  [37] .rela.debug_frame RELA            0000000000000000 8d58d0 006de0 18   I 52  36  8
  [38] .orc_unwind_ip    PROGBITS        0000000000000000 8dc6b0 003468 04   A  0   0  1
  [39] .rela.orc_unwind_ip RELA            0000000000000000 8dfb18 013a70 18   I 52  38  8
  [40] .orc_unwind       PROGBITS        0000000000000000 8f3588 004e9c 06   A  0   0  1
  [41] __mcount_loc      PROGBITS        0000000000000000 8f8428 001208 08   A  0   0  8
  [42] .rela__mcount_loc RELA            0000000000000000 8f9630 003618 18   I 52  41  8
  [43] __param           PROGBITS        0000000000000000 8fcc48 0000f0 00   A  0   0  8
  [44] .rela__param      RELA            0000000000000000 8fcd38 000240 18   I 52  43  8
  [45] .modinfo          PROGBITS        0000000000000000 8fcf80 0002be 00   A  0   0 16
  [46] .data.once        PROGBITS        0000000000000000 8fd23e 000001 00  WA  0   0  1
  [47] .note.Linux       NOTE            0000000000000000 8fd240 000034 00   A  0   0  4
  [48] .gnu.linkonce.this_module PROGBITS        0000000000000000 8fd280 000380 00  WA  0   0 64
  [49] __versions        PROGBITS        0000000000000000 8fd600 002d80 00   A  0   0 16
  [50] .note.gnu.build-id NOTE            0000000000000000 900380 000018 00   A  0   0  4
  [51] .note.GNU-stack   PROGBITS        0000000000000000 900398 000000 00      0   0  1
  [52] .symtab           SYMTAB          0000000000000000 900398 0070f8 18     54 728  8
  [53] .shstrtab         STRTAB          0000000000000000 907490 000323 00      0   0  1
  [54] .strtab           STRTAB          0000000000000000 9077b3 00771e 00      0   0  1
Key to Flags:
  W (write), A (alloc), X (execute), M (merge), S (strings), I (info),
  L (link order), O (extra OS processing required), G (group), T (TLS),
  C (compressed), x (unknown), o (OS specific), E (exclude),
  l (large), p (processor specific)

[1] https://llvm.org/pr42289

dileks commented 5 years ago

Just looked after the alignment sizes:

[ arch/x86/crypto/crct10dif-pcl-asm_64.S. ]
.section .rodata.cst32.byteshift_table, "aM", @progbits, 32
.align 16

Should this be 32?

UPDATE: "crypto: x86 - make constants readonly, allow linker to merge them"

[1] https://git.kernel.org/linus/0974037fc55c

dileks commented 5 years ago

@nickdesaulniers

I can boot with your patch, but I see this:

[    1.848782] sysfs: cannot create duplicate filename '/module/i915/sections/.rodata.cst8'

$ cd drivers/gpu/drm/i915/

$ readelf -WS i915.ko | grep rodata.\cst8
  [60] .rodata.cst8      PROGBITS        0000000000000000 362fd90 000020 08  AM  0   0  2
  [77] .rodata.cst8      PROGBITS        0000000000000000 36340c4 000008 08  AM  0   0  1
dileks commented 5 years ago

This eliminates all sysfs:warnings for me:

diff --git a/scripts/module-common.lds b/scripts/module-common.lds
index c933830b4b62..3bbb81441bf3 100644
--- a/scripts/module-common.lds
+++ b/scripts/module-common.lds
@@ -24,6 +24,7 @@ SECTIONS {

        __jump_table            0 : ALIGN(8) { KEEP(*(__jump_table)) }

+       .rodata.cst8            0 : { *(.rodata.cst8) }
        .rodata.cst16           0 : { *(.rodata.cst16) }
        .rodata.cst32           0 : { *(.rodata.cst32) }
 }
nickdesaulniers commented 5 years ago

@dileks started this thread, with a helpful find: hhttps://lore.kernel.org/r/CA+icZUX3EOhrUp0Afbo_fK9rb5AbXjbaBFwhj1qmBaHom1b3MA@mail.gmail.com/

(I guess that would handle the cst32 case only though?)

MaskRay commented 5 years ago

Whether or not the kernel correctly uses .rodata.cst* etc, this exposes one place where lld can improve its merging efficiency: sections with the same name but different alignments can also be merged. This was fixed by D63432/r365015. I think that should effectively fix the issue.

If the kernel tool assumes there is only one .rodata.cst32, I think it'd better get rid of that assumption. If you pass -Wl,-O0, one may expect SHF_MERGE sections should not merged[1], a natural consequence is that there may be multiple .rodata.cst32 with varying alignments.

[1]: lld currently forces -O1 in -r mode. This is to work around some tools:

tools like (llvm-)?dwarfdump get confused when they see two .debug_str. We could have separate logic for combining SHF_MERGE sections based both on their name and sh_entsize,

It'd be good to get rid of this hack at some point.

[2]: ld.bfd and gold appear to leave just one output section. This choice is less ideal if there are sections with the same name but different sh_entsize

dileks commented 5 years ago

@MaskRay

This looks good to me. Thanks.

$ ld.lld-9 --version
LLD 9.0.0 (git://github.com/llvm/llvm-project 9854d771bd11629df5ea21403457dcc7c7b84a7c) (compatible with GNU linkers)

$ cd fs/nfsd/

$ for o in $(ls *.o) ; do echo [ $o ] ; readelf -WS $o | grep rodata\.cst32 ; done
[ auth.o ]
[ blocklayout.o ]
[ blocklayoutxdr.o ]
[ export.o ]
  [10] .rodata.cst32     PROGBITS        0000000000000000 005720 000020 20  AM  0   0  4
[ lockd.o ]
[ nfs2acl.o ]
[ nfs3acl.o ]
[ nfs3proc.o ]
  [12] .rodata.cst32     PROGBITS        0000000000000000 004090 000020 20  AM  0   0 16
[ nfs3xdr.o ]
[ nfs4acl.o ]
[ nfs4callback.o ]
[ nfs4idmap.o ]
[ nfs4layouts.o ]
[ nfs4proc.o ]
[ nfs4recover.o ]
[ nfs4state.o ]
[ nfs4xdr.o ]
[ nfscache.o ]
[ nfsctl.o ]
[ nfsd.o ]
  [59] .rodata.cst32     PROGBITS        0000000000000000 7cd990 000040 20  AM  0   0 16
[ nfsfh.o ]
[ nfsproc.o ]
[ nfssvc.o ]
[ nfsxdr.o ]
[ stats.o ]
[ trace.o ]
[ vfs.o ]

ReadELF Outputs with broken LLD see [1].

[1] https://llvm.org/pr42289

tpimh commented 5 years ago

Fixed by D63432/rLLD365015.

nathanchance commented 5 years ago

Reverted in r365048.

dileks commented 5 years ago

My Linux-kernel booted with no sysfs warnings:

$ cat /proc/version 
Linux version 5.1.16-1-amd64-cbl-asmgoto (sedat.dilek@gmail.com@iniza) (ClangBuiltLinux clang version 9.0.0 (git://github.com/llvm/llvm-project 9854d771bd11629df5ea21403457dcc7c7b84a7c) (based on LLVM 9.0.0svn)) #1~buster+dileks1 SMP 2019-07-03

$ sudo LC_ALL=C dmesg -T | grep sysfs

The first time I used a selfmade llvm-toolchain built with tc-build [1].

[1] https://github.com/ClangBuiltLinux/tc-build

MaskRay commented 5 years ago

Fixed by D64200/rLLD365139.

I still believe, if possible, the kernel tool should allow multiple .rodata.cst{16,32} with different alignments in a partially linked object file. That may have a very low priority, though.

dileks commented 5 years ago

Looks good to me:

$ clang-9 --version
ClangBuiltLinux clang version 9.0.0 (git://github.com/llvm/llvm-project 5c4bbc274663450b2913154de51995dd66457048) (based on LLVM 9.0.0svn)
Target: x86_64-unknown-linux-gnu
Thread model: posix
InstalledDir: /usr/bin

$ ld.lld-9 --version
LLD 9.0.0 (git://github.com/llvm/llvm-project 5c4bbc274663450b2913154de51995dd66457048) (compatible with GNU linkers)

$ cd linux-5.1.16/fs/nfsd/

$ for o in $(ls *.o) ; do echo [ $o ] ; readelf -WS $o | grep rodata\.cst32 ; done
[ auth.o ]
[ blocklayout.o ]
[ blocklayoutxdr.o ]
[ export.o ]
  [10] .rodata.cst32     PROGBITS        0000000000000000 005720 000020 20  AM  0   0  4
[ lockd.o ]
[ nfs2acl.o ]
[ nfs3acl.o ]
[ nfs3proc.o ]
  [12] .rodata.cst32     PROGBITS        0000000000000000 004090 000020 20  AM  0   0 16
[ nfs3xdr.o ]
[ nfs4acl.o ]
[ nfs4callback.o ]
[ nfs4idmap.o ]
[ nfs4layouts.o ]
[ nfs4proc.o ]
[ nfs4recover.o ]
[ nfs4state.o ]
[ nfs4xdr.o ]
[ nfscache.o ]
[ nfsctl.o ]
[ nfsd.o ]
  [59] .rodata.cst32     PROGBITS        0000000000000000 7cd940 000040 20  AM  0   0 16
[ nfsfh.o ]
[ nfsproc.o ]
[ nfssvc.o ]
[ nfsxdr.o ]
[ stats.o ]
[ trace.o ]
[ vfs.o ]
nickdesaulniers commented 5 years ago

Thanks @MaskRay for the updated fix, and @dileks for reporting and testing the fixes!