llvmbot
commented
1 week ago @llvm/issue-subscribers-backend-risc-v
Author: Kito Cheng (kito-cheng)
Unfortunately, whole register move instructions depend on `vtype`*1, which means they will cause an illegal instruction exception if VILL=1. This is generally not a problem, as VILL is set to 0 after any valid `vsetvli` instruction, so it’s usually safe unless the user executes a whole vector register move very early in the program.
However, the situation changed after the Linux kernel applied a patch[2] that sets VILL=1 after any system call. So, if we try to execute a whole register move after a system call, it will cause an illegal instruction exception. This can be difficult to detect, as the system call may not be invoked immediately; it might be deeply nested in a call chain, such as within `printf`. Unfortunately, this change has already shipped with Linux kernel 6.5, which was released on August 28, 2023.
I'm not sure if it's reasonable to ask the Linux kernel maintainers to fix this by keeping VILL consistent across system calls.
An alternative approach is to address this issue on the toolchain side by requiring at least one valid `vsetvli` instruction before any whole register move. This might be an ugly workaround, but it’s probably the simplest way to resolve the issue. I also realized this might be a better solution since the psABI specifies that `VTYPE` is NOT preserved across function calls. This means we can’t guarantee that VILL is not 1 at the function entry, so placing a `vsetvli` instruction right after the function call may be necessary.
Testcase:
```c
#include <riscv_vector.h>
void bar() __attribute__((riscv_vector_cc));
vint32m1_t foo(vint32m1_t a) {
bar(); // We never know bar will call system call inside or not.
return a;
}
```
Generated asm with `clang -target riscv64-unknown-elf -S -O3 -march=rv64gcv`:
```asm
...
.type foo,@function
.variant_cc foo
foo: # @foo
# %bb.0: # %entry
addi sp, sp, -16
sd ra, 8(sp) # 8-byte Folded Spill
vmv1r.v v24, v8
call bar
vmv1r.v v8, v24
ld ra, 8(sp) # 8-byte Folded Reload
addi sp, sp, 16
ret
...
```
And the compiler could emits code like below to fix this issue:
```asm
...
.type foo,@function
.variant_cc foo
foo: # @foo
# %bb.0: # %entry
addi sp, sp, -16
sd ra, 8(sp) # 8-byte Folded Spill
vsetivli x0, 0, e8, m1, ta, ma # Need vsetvli to make VILL=0 here
vmv1r.v v24, v8
call bar
vsetivli x0, 0, e8, m1, ta, ma # Need vsetvli to make VILL=0 here
vmv1r.v v8, v24
ld ra, 8(sp) # 8-byte Folded Reload
addi sp, sp, 16
ret
...
```
NOTE: We have hit this issue within our internal spec run.
`*1` That change[1] is made *AFTER* 1.0...
[1] https://github.com/riscvarchive/riscv-v-spec/commit/856fe5bd1cb135c39258e6ca941bf234ae63e1b1
[2] https://github.com/torvalds/linux/commit/9657e9b7d2538dc73c24947aa00a8525dfb8062c
kito-cheng
wangpc-pp
preames
topperc
aswaterman
luismarques
dzaima
Unfortunately, whole register move instructions depend on
vtype*1, which means they will cause an illegal instruction exception if VILL=1. This is generally not a problem, as VILL is set to 0 after any validvsetvliinstruction, so it’s usually safe unless the user executes a whole vector register move very early in the program.However, the situation changed after the Linux kernel applied a patch[2] that sets VILL=1 after any system call. So, if we try to execute a whole register move after a system call, it will cause an illegal instruction exception. This can be difficult to detect, as the system call may not be invoked immediately; it might be deeply nested in a call chain, such as within
printf. Unfortunately, this change has already shipped with Linux kernel 6.5, which was released on August 28, 2023.I'm not sure if it's reasonable to ask the Linux kernel maintainers to fix this by keeping VILL consistent across system calls.
An alternative approach is to address this issue on the toolchain side by requiring at least one valid
vsetvliinstruction before any whole register move. This might be an ugly workaround, but it’s probably the simplest way to resolve the issue. I also realized this might be a better solution since the psABI specifies thatVTYPEis NOT preserved across function calls. This means we can’t guarantee that VILL is not 1 at the function entry, so placing avsetvliinstruction right after the function call may be necessary.Testcase:
Generated asm with
clang -target riscv64-unknown-elf -S -O3 -march=rv64gcv:And the compiler could emits code like below to fix this issue:
NOTE: We have hit this issue within our internal spec run.
*1That change[1] is made AFTER 1.0... [1] https://github.com/riscvarchive/riscv-v-spec/commit/856fe5bd1cb135c39258e6ca941bf234ae63e1b1[2] https://github.com/torvalds/linux/commit/9657e9b7d2538dc73c24947aa00a8525dfb8062c