filecoin-project / ec-gpu

OpenCL code generator for finite-field arithmetic over arbitrary prime fields
Other
91 stars 61 forks source link

feat: add FFT and Multiexp kernels #14

Closed vmx closed 2 years ago

vmx commented 2 years ago

This commit copies the FFT and Multiexp kernels from bellperson into this library. The goal is to have general purpose implementation of those, for both CPU and GPU.

The implementations work for both CUDA and OpenCL. They can be enabled with the corresponding features.


This PR looks like there would be a lot of new code to review, but that isn't actually the case. It is mostly a refactor, which moves code from bellperson into this library. In order to make this review easier, I propose diffing files from this PR and bellperson. Here is a list of which files should be compared.

To see all files that changed, use git show --stat. Below I'll mention all of those files from that list and how to review them the easiest way.

Have a checkout of this directory next to a checkout of bellperson on its master branch. I've created a list of files you can use to diff between. Prefix those two paths with the diff tool of your choice and you should have a better time reviewing.

ec-gpu/ec-gpu-gen/build.rs bellperson/build.rs
ec-gpu/ec-gpu-gen/src/cl/fft.cl bellperson/src/gpu/fft/fft.cl
ec-gpu/ec-gpu-gen/src/cl/multiexp.cl bellperson/src/gpu/multiexp/multiexp.cl

# For the code.
ec-gpu/ec-gpu-gen/src/fft.rs bellperson/src/gpu/fft.rs
# For the test cases.
ec-gpu/ec-gpu-gen/src/fft.rs bellperson/src/domain.rs

ec-gpu/ec-gpu-gen/src/fft_cpu.rs bellperson/src/domain.rs

# The `parallel_multiexp` is based on the `et cpu_acc = pool.scoped(|s| {` part in the bellperson code.
ec-gpu/ec-gpu-gen/src/multiexp.rs bellperson/src/gpu/multiexp.rs
# For the `gpu_multiexp_consistency()` test
ec-gpu/ec-gpu-gen/src/multiexp.rs bellperson/src/multiexp.rs

# The `test_with_bls12` is also part of this bellperson source file, my diff tool didn't recognize that properly.
ec-gpu/ec-gpu-gen/src/multiexp_cpu.rs bellperson/src/multiexp.rs

ec-gpu/ec-gpu-gen/src/program.rs bellperson/src/gpu/program.rs
ec-gpu/ec-gpu-gen/src/threadpool.rs bellperson/src/multicore.rs

Large parts of lib.rs and nvidia.rs were moved into sources.rs. To verify that, you can use Git's --color-moved and just compare those files where the movement happened (or use copies from master and compare those).

git show --color-moved -- ec-gpu-gen/src/lib.rs ec-gpu-gen/src/source.rs
git show --color-moved -- ec-gpu-gen/src/nvidia.rs ec-gpu-gen/src/source.rs

Files that are part of that PR, which we haven't discussed above, which need manual review:

ec-gpu/.circleci/config.yml
ec-gpu/README.md
ec-gpu/ec-gpu-gen/Cargo.toml
ec-gpu/ec-gpu-gen/src/error.rs
ec-gpu/rust-toolchain

One TODO item is updating the README, to reflect those additions better. I'll do that, but I thought the review can already start, as I expect it to take quite some time.

The CI failure seems to be a CircleCI env issue. That will be a tough one to debug.