gnzlbg
commented
6 years ago I can mentor you. If it's your first contribution it is easier if you report "regularly" or just open a PR so that we can track your progress and help if you get stuck. This is, in a nutshell, the list of all steps necessary. It looks like a lot of steps, but these steps are very very small. In your case, the step number is a bit larger because you want to add intrinsics for a new feature that we haven't white-listed yet.
- Step 1: White-list the feature in rustc: if the compiler does not recognize the feature name, it will (or should) give you an error, so the first step is to teach the compiler the feature name. This is a one line change in the compiler, the difficult part is coming up with the feature name.
- Find the name of the cpuid bit feature / features. For step 2 you will also need to know how to query them, for x86 this is which bits you need to read from which registers after performing a particular query using the
cpuidinstruction. This page tells you everything: https://en.wikipedia.org/wiki/CPUID - Find the name of the LLVM backend feature / feature names -
llc --mcpu=helpgives you a list of all supported cpus by your LLVM version. Choose one cpu that has the feature andllc --mcpu=<cpu_here> --mattr=helpwill tell you all the backend features that the CPU supports. Find the name of the one you care about. - As a good rule of thumb: name the feature like LLVM does, but in the PR provide the name of the cpuid feature, and maybe also how clang and gcc name it. If they all match there is nothing to discuss. If Intel names it differently than LLVM the name will be clarified in the PR to rustc.
- white-list the feature in rustc by adding it to this list: https://github.com/rust-lang/rust/blob/36b66873187e37a9d79adad89563088a9cb86028/src/librustc_trans/llvm_util.rs#L88
- submit a PR to rustc - you can work on the Step 2 next, but for steps 3..5 this PR will need to have landed, and a new nightly be produced, before your code will pass the tests. Note, after a rustc PR is accepted, it takes from a couple of hours to a couple of days to actually get merged, depending on how many PRs are on flight.
- Find the name of the cpuid bit feature / features. For step 2 you will also need to know how to query them, for x86 this is which bits you need to read from which registers after performing a particular query using the
- Step 2: implementing run-time feature detection for the intrinsic in
stdsimd: to use intrinsics that require the feature, the CPU must support it, so in this step you need to send a PR tostdsimdthat includes the code that detects whether the feature is enabled or not, and if its enabled, activates the feature. This is a two-liner change.- you need to add the intrinsic to the supported intrinsics for the architecture in the
detectmodule, forx86that would be here: https://github.com/rust-lang-nursery/stdsimd/blob/master/stdsimd/arch/detect/arch/x86.rs#L28 - you need to check whether the intrinsic is enabled at run-time, and if that's the case, enable the feature. For x86's you need to check whether the cpuid bit of the intrinsic is set and if so, enable the intrinsic like the other code here does: https://github.com/rust-lang-nursery/stdsimd/blob/master/stdsimd/arch/detect/os/x86.rs#L125
- you need to update some run-time feature detection tests (one liner each):
- here: https://github.com/rust-lang-nursery/stdsimd/blob/master/stdsimd/arch/detect/os/x86.rs#L239
- here: https://github.com/rust-lang-nursery/stdsimd/blob/master/stdsimd/arch/detect/os/linux/cpuinfo.rs#L134
- here: https://github.com/rust-lang-nursery/stdsimd/blob/master/crates/stdsimd/tests/cpu-detection.rs#L45
- Send a PR to
stdsimdwith this step.
- you need to add the intrinsic to the supported intrinsics for the architecture in the
- Step 3: Implementing the intrinsics that you care about. This is where the meat is.
- First, you need to come up with a place to implement the feature inside the
coresimd/x86module. If the feature is its own self-standing feature, you probably want to add a new module there called<featurename>.rs(check how rdtsc does it: https://github.com/rust-lang-nursery/stdsimd/blob/master/coresimd/x86/rdtsc.rs) - You now have to add the intrinsic/intrinsics to this module. Check how rdtsc does it (https://github.com/rust-lang-nursery/stdsimd/blob/master/coresimd/x86/rdtsc.rs). In a nutshell:
- You typically have to call an LLVM intrinsic function. To be able to call it, you need to declare those functions as
externand uselink_llvm_intrinsicto give it the LLVM function name. Like here: https://github.com/rust-lang-nursery/stdsimd/blob/master/coresimd/x86/rdtsc.rs#L48 . Finding the names of the llvm functions that you want to call can be tricky. A good approach is to search for the C intrinsic in https://github.com/llvm-mirror/clang/tree/master/lib/Headers to see how clang implements them. If it uses a builtin, you can search for the code-gen tests that make sure that it generates the right call to the corresponding LLVM function in clang and/or LLVM source files. For example, I just searched in the clang repo for rdtsc and landed here: https://github.com/llvm-mirror/clang/blob/a38ba9770a70056f9fdd6c71f819e5db45a105e4/test/CodeGen/rd-builtins.c#L15 Thellvm.x86....is the name of the intrinsic one is looking for. - You then need to write a Rust wrapper over it. This is the function that will be exposed by the library. They look like this (https://github.com/rust-lang-nursery/stdsimd/blob/master/coresimd/x86/rdtsc.rs#L22). In a nutshell:
- These functions need to be marked:
#[inline] - These functions must be compiled with the required feature/features enabled:
#[target_feature(enable = "featurename")] - You need to add a test that disassembles the function at run-time and tests whether it contains the CPU instruction that the intrinsic should generate. The
assert_instr(<instruction_name>)procedural macro does this for you. The #[cfg_attr(test, ...)] just makes sure that these tests are only generated when compiling the tests of the library. - You need to add a run-time test that verifies that the intrinsic produced correct results. See how rdtsc does it, but in a nutshell we have a procedural macro
#[simd_test = "<feature_name>"]that makes sures that the test only runs if the CPU where the library is being tested actually supports the feature. - Intrinsics need docs: one sentence stating what the intrinsic does is fine (at most, some explanation in the order of what clang does is fine). This library is not a guide/tutorial about how to use intrinsics.
- These functions need to be marked:
- First, you need to come up with a place to implement the feature inside the
- Step 4: get test to pass on CI, you will need to enable travis for your fork of
stdsimdfor this. Test might fail for a variety of reasons, and we have seen many of them. So if you get stuck here just go directly to step 5 and we'll take a look at it. - Step 5: submit a PR to stdsimd with the implementation of the intrinsic! If you get stuck, encounter unexpected issues, or just want some feedback, just submit a PR early so that we can give you feedback as you progress.
Also, if you want to add newer intrinsics for a particular feature, you don't need Step 1 and 2, and can go directly to Steps 3,4,5.
alexcrichton
jasondavies
What needs to be done to support Intel's SHA extensions?
Happy to help if someone is willing to mentor/guide me!
I would also note that there are equivalent/similar extensions for ARM and POWER8. But getting support for the Intel extensions would be a great start; they are already supported on AMD Ryzen.