RustCrypto / hashes

Collection of cryptographic hash functions written in pure Rust
1.82k stars 247 forks source link

Cannot build on x86_64-unknown-uefi target #446

Closed architector1324 closed 1 year ago

architector1324 commented 1 year ago
cargo build --target=x86_64-unknown-uefi
...
error: could not compile `sha2`
LLVM ERROR: Do not know how to split the result of this operator!
tarcieri commented 1 year ago

Seems like a similar issue to https://github.com/RustCrypto/AEADs/issues/498: we make use of AVX2 and SHA-NI when available.

Try enabling the force-soft feature.

@newpavlov perhaps we should have cpufeatures auto-disable CPU intrinsics on UEFI targets

architector1324 commented 1 year ago

@tarcieri Thanks! Using force-soft feature solved this problem.

newpavlov commented 1 year ago

@tarcieri It's not only UEFI targets, in the linked AEAD issue a custom target derived from x86_64-unknown-none was used. We need to find whether it's an LLVM bug or if there is a reason for such behavior. If it's indeed a compiler bug, then I am not sure we should introduce workarounds for it into cpufeatures. I probably will create an LLVM issue for it a bit later.

tarcieri commented 1 year ago

Here's another report of a similar problem... seems like we should do something about this:

https://github.com/RustCrypto/SSH/issues/57

newpavlov commented 1 year ago

It looks like LLVM can not use XMM registers because x86_64-unknown-uefi enables the soft-float target feature, thus it tries to scalarize the SHA instructions, but it's unable to do so.

I think that CPUID would've reported that SSE and as extension SHA is not available, so our XMM-based code would've been unreachable. Removing +soft-float on line 31 here results in expected assembly. Unfortunately, Rust does not allow us to disable soft floats locally for a function and AFAIK does not allow detection if it's enabled or not in cfg statements. Also we get hit by deficiency of the Rust target feature system. It currently only supports two states in library code: "target feature is enabled" (+sha) and "target feature may exist" (?sha), while we also need "target feature is disabled" (-sha).

For now, the most practical approach would be to disable runtime target feature detection for selected targets, but it's nothing more than a workaround for Rust issues...

tarcieri commented 1 year ago

Perhaps we can compile a list of commonly used problematic platforms for now, and then separately work on trying to get the issue addressed upstream in LLVM

newpavlov commented 1 year ago

Apart from the bad diagnostics, I think LLVM behaves more or less correctly. Arguably, soft-float should not be a "positive" target feature, since it's not additive (i.e. +sse and +soft-float conflict with each other), but I don't believe we will be able to push for it to change. The issue is mostly on the Rust side, cpufeatures should be able to detect target features disabled at compile time, i.e. if a user compiles an application with explicit -avx2 we should remove AVX2-based code branches, even though in theory the application binary may be launched on a CPU with AVX2 support.

yaoxin1995 commented 1 year ago

Hey, I am not sure where to add force-soft feature.

Do I need to add it to

{
  "llvm-target": "x86_64-unknown-none",
  "data-layout": "e-m:e-i64:64-f80:128-n8:16:32:64-S128",
  "linker-flavor": "gcc",
  "target-endian": "little",
  "target-pointer-width": "64",
  "target-c-int-width": "32",
  "arch": "x86_64",
  "os": "none",
  "disable-redzone": true,
  **"features": "-mmx,-sse,-avx,+soft-float,force-soft",**
  "panic-strategy": "abort"
}

Or in here:

sha2 = { version = "0.10.6", default-features = false,  features = ["force-soft"]}
newpavlov commented 1 year ago

@yaoxin1995 It's the latter.

tarcieri commented 1 year ago

This should be addressed by cpufeatures v0.2.7 which will force the soft implementation on freestanding UEFI targets automatically