Illegal instruction in debug mode when processing MemoryOrdering::from_llvm

[sunziping2016]

When processing the following instruction:

[2021-11-17T13:52:02Z DEBUG llvm_ir::instruction] Processing instruction "  fence seq_cst, !dbg !164"

MemoryOrdering::from_llvm get a parameter of value 3, which corresponding to no variant of the enum, causing illegal instruction.

This bug can only be triggered in debug mode.

LLVM version 13.0.0

OS: ArchLinux

[cdisselkoen]

I'm having trouble reproducing this exact error, but there's definitely some problems going on with fence instructions and their MemoryOrderings, not just for LLVM 13 but for many LLVM versions. I'll continue investigating but just posting this update here

[sunziping2016]

I'm having trouble reproducing this exact error, but there's definitely some problems going on with fence instructions and their MemoryOrderings, not just for LLVM 13 but for many LLVM versions. I'll continue investigating but just posting this update here

Thanks for the friendly and helpful reply. I am really sorry that I can't give a good minimum reproduction because my project has a big code base. If you need, I can send you the LLVM bitcode that may trigger this problem by loading it.

[cdisselkoen]

That would be helpful for sure!

[sunziping2016]

The first-hand sources are available at my own file cloud. The link should be valid for months but not forever.

Three files are equivalent to each other. They are extracted or compiled from NGINX.

The problem originated from following IR snippet:

define dso_local void @ngx_explicit_memzero(i8* %buf, i64 %n) #0 !dbg !3950 {
entry:
  %buf.addr = alloca i8*, align 8
  %n.addr = alloca i64, align 8
  store i8* %buf, i8** %buf.addr, align 8
  call void @llvm.dbg.declare(metadata i8** %buf.addr, metadata !3953, metadata !DIExpression()), !dbg !3954
  store i64 %n, i64* %n.addr, align 8
  call void @llvm.dbg.declare(metadata i64* %n.addr, metadata !3955, metadata !DIExpression()), !dbg !3956
  %0 = load i8*, i8** %buf.addr, align 8, !dbg !3957
  %1 = load i64, i64* %n.addr, align 8, !dbg !3958
  call void @llvm.memset.p0i8.i64(i8* align 1 %0, i8 0, i64 %1, i1 false), !dbg !3959
  fence seq_cst, !dbg !3960
  ret void, !dbg !3961
}

which corresponds to this:

https://github.com/nginx/nginx/blob/3334585539168947650a37d74dd32973ab451d70/src/core/ngx_string.c#L2078-L2083

The definitions of ngx_memory_barrier may be one of these, generated by grep.

src/os/unix/ngx_gcc_atomic_amd64.h
80:#define ngx_memory_barrier()    __asm__ volatile ("" ::: "memory")

src/os/unix/ngx_atomic.h
37:#define ngx_memory_barrier()        AO_nop()
65:#define ngx_memory_barrier()        __sync_synchronize()
117:#define ngx_memory_barrier()        OSMemoryBarrier()
153:#define ngx_memory_barrier()        __asm (".volatile"); __asm (".nonvolatile")
194:#define ngx_memory_barrier()        __asm (".volatile"); __asm (".nonvolatile")
301:#define ngx_memory_barrier()

src/os/unix/ngx_gcc_atomic_x86.h
124:#define ngx_memory_barrier()    __asm__ volatile ("" ::: "memory")

src/os/unix/ngx_sunpro_atomic_sparc64.h
56:#define ngx_memory_barrier()                                                  \

src/os/unix/ngx_gcc_atomic_sparc64.h
74:#define ngx_memory_barrier()                                                  \
79:#define ngx_memory_barrier()   __asm__ volatile ("" ::: "memory")

src/os/unix/ngx_gcc_atomic_ppc.h
83:#define ngx_memory_barrier()                                                  \
86:#define ngx_memory_barrier()   __asm__ volatile ("" ::: "memory")
146:#define ngx_memory_barrier()                                                  \
149:#define ngx_memory_barrier()   __asm__ volatile ("" ::: "memory")

src/os/win32/ngx_atomic.h
47:#define ngx_memory_barrier()

Clang-13 chose __sync_synchronize() as the expansion of this macro, which is compiled into fence instruction.

[sunziping2016]

To clarify, "can only be triggered in debug mode" means triggered in Rust debug profile not compiling the C codes in debug mode.

When processing the following instruction:

[2021-11-17T13:52:02Z DEBUG llvm_ir::instruction] Processing instruction "  fence seq_cst, !dbg !164"

MemoryOrdering::from_llvm get a parameter of value 3, which corresponding to no variant of the enum, causing illegal instruction.

This bug can only be triggered in debug mode.

LLVM version 13.0.0

OS: ArchLinux

[cdisselkoen]

Thanks for the info!

I believe this is an upstream issue; I've filed https://gitlab.com/taricorp/llvm-sys.rs/-/issues/23 with more details. We'll wait and see what the response is there.

[sunziping2016]

Hahahaha. I filed https://gitlab.com/taricorp/llvm-sys.rs/-/issues/22 some day before. And they said it might be your problem.

[cdisselkoen]

Sure. There might be problems both places. I created a different reproducer in https://gitlab.com/taricorp/llvm-sys.rs/-/issues/23 which shows that there is definitely something wrong either in llvm-sys or in LLVM itself. Once that's fixed we can see if there is also a problem here

[cdisselkoen]

In #44 a kind soul investigated and diagnosed this as a bug in LLVM, and also filed an upstream bug and corresponding PR. I'll go ahead and close this in favor of #44.