Self Proposed Open Task: Implement a polynomial repo integrating Icicle and sppark

[cyl19970726]

[!IMPORTANT] A proposal will go through a review process by a PSE to ensure the quality of the task and alignment with the acceleration program mission. Please be patient and wait for the review to complete.

Open Task RFP for Langlands

Executive Summary

• Project Overview: Implement KZG over a polynomial repo and integrate Sppark and Icicle

Project Details

• Motivation:
  • The different high-performance hardware-accelerated versions of the same proof system, developed by various teams, are often proprietary because, for most teams, it is a vital weapon that allows them to survive in commercial competition. This situation is similar to the existence of multiple GPU versions of a game like Halo2, with examples including the Scroll, DelphinusLab, Tachyon, and Ingoyama versions. Each version, developed by different teams, brings unique features or performance enhancements that provide a competitive edge in the market. But I don’t think this is the way to maximize value because developers often do a lot of duplication of work.
  • In the future, there may be more such general acceleration libraries that not only work on GPUs but also on ASICs and FPGAs. So for most teams, it is impossible for them to spend a lot of time integrating so much hardware. acceleration libraries and compare their performance differences one by one. We need to add an intermediate layer to the proof system and the hardware acceleration library so that most teams that develop proof systems only need to integrate this middle layer to run on different hardware flexibly. Switch between acceleration plans. Zpu is a potential solution, but we hope to do this in the upper polynomial layer of ZPU developed by Ingoyama, because for all proof systems, they essentially end up operating on polynomials, so we hope to provide a useful polynomial library and integrate enough common hardware acceleration libraries on this library.
  • We currently lack many ZKP hardware acceleration engineers because there are very few people with the knowledge base of both ZKP and hardware acceleration, and the learning fields in these two fields are also very steep. We need to provide something like https://learn.0xparc.org/ /halo2/’s courses to help more developers enter this area.
• Scope of Work:
  • Choose a polynomial library that is general enough, and we decide to select https://github.com/arkworks-rs/algebra/tree/master/poly as the repo to develop.
  • Implement KZG over this polynomial repo and integrate Sppark and Icicle
  • We plan to output some educational content about developing the MSM algorithm using Cuda and compare it with different MSM-CUDA versions developed by different awesome teams.
• Expected Outcomes:
  • Implement KZG over this polynomial repo and integrate Sppark and Icicle.
  • Support CQ lookup argument
  • Write an article to summarize the similarities and differences between Icicle and Sppark in the implementation of the MSM algorithm and compare the performance data.

Qualifications

• Skills Required:
  • Good rust development capabilities
  • A deep understanding of snark
  • A certain understanding of GPU programming
• Preferred Qualifications:
  • Excellent document writing skills

Administrative Details

• Estimated Project Duration: 5 weeks
• Project Complexity: Medium
  • because doing the job requires some understanding of a variety of technologies. However, you don’t need to have an in-depth understanding of every technical field. More work is to find some better technical solutions on the market and combine them.

Additional Information

• Reference Material:
  • We ultimately hope that this project can integrate more lookup arguments, more commit schemes, and more hardware acceleration libraries so that developers can develop a new proof system easily with better performance by using this polynomial repo.
  • icicle: https://github.com/ingonyama-zk/icicle.git
  • sppark: https://github.com/supranational/sppark.git

[NOOMA-42]

Thanks for raising this proposal, would you be able to leave how long your open task could last in Estimated Project Duration?

[cyl19970726]

Thanks for raising this proposal, would you be able to leave how long your open task could last in Estimated Project Duration?

Thanks for your advice, have updated the 'Estimated Project Duration'.

[NOOMA-42]

Would you be interested in integrate this hardware acceleration to halo2? There's only PCS is generic in halo2, and not MSM, FFT. Sometimes this is integrated with PCS. It'll be nice to have PCS trait which is easier for hardware acceleration.

Thanks for raising this proposal, would you be able to leave how long your open task could last in Estimated Project Duration?

Thanks for your advice, have updated the 'Estimated Project Duration'.

[cyl19970726]

For MSM, I plan to add a tait Commit and change the trait Params from pub trait Params<'params, C: CurveAffine>: Sized + Clone to pub trait Params<'params, C: CurveAffine, CS: Commit>: Sized + Clone, Then restrict the commit_langage() function to use Commit:commit(). And do the same modification for ParamsProver trait. In this way, different MSM algorithms can be called by the entire proof system as long as they implement the Commit trait. We can also easily integrate different GPU and CPU versions of MSM algorithms.

[cyl19970726]

This does not involve a lot of code, so it will be completed by February 15th, including abstracting the Commit Trait and integrating the MSM algorithm of Iccle and Sppark.

[NOOMA-42]

it would be more usable/referable to base the interface on https://github.com/privacy-scaling-explorations/halo2curves/pull/107 to see if there was a way to integrate these two libraries into this new interface, and then in the process if there were any suggestions for feedback to the PR. If there are any suggestions, you can feed back to this PR.

The final product could be:

1. based on icicle/sppark and then implement the MsmAccel trait
2. integrate MsmAccel into a library (e.g. halo2, their own kzg lib) and show how to switch between different implementations.

Later, they will also integrate MsmAccel into halo2, so if they can try to integrate halo2, it will be helpful, but I don't feel that we have to choose halo2, the focus is still on the first point.

This does not involve a lot of code, so it will be completed by February 15th, including abstracting the Commit Trait and integrating the MSM algorithm of Iccle and Sppark.

[cyl19970726]

It would be more usable/referable to base the interface on privacy-scaling-explorations/halo2curves#107 to see if there was a way to integrate these two libraries into this new interface, and then in the process if there were any suggestions for feedback to the PR. If there are any suggestions, you can feed back to this PR.

The final product could be:

1. based on icicle/sppark and then implement the MsmAccel trait
2. integrate MsmAccel into a library (e.g. halo2, their own kzg lib) and show how to switch between different implementations.

Later, they will also integrate MsmAccel into halo2, so if they can try to integrate halo2, it will be helpful, but I don't feel that we have to choose halo2, the focus is still on the first point.

This does not involve a lot of code, so it will be completed by February 15th, including abstracting the Commit Trait and integrating the MSM algorithm of Iccle and Sppark.

I think this is more meaningful than my original Idea. I initially set up three milestones:

1. integrate icicle(implement the MsmAccel trait for icicle-msm)(1 week)
2. integrate sppark(implement the MsmAccel trait for sppark-msm) (1 week)
3. integrate MsmAccel trait into Halo2 (1 week)

[NOOMA-42]

LGTM, please kindly send out your proposal if you'd like to work on this

[Vap0RSh4wN]

Hello! As the current time has been in round2, I would like to ask you whether this task is still open now? And where has it gone? I am very interested, please feel free to contact me at vaporshawnn@gmail.com!

[NOOMA-42]

Thanks for your interest, but right now it's been worked on

Hello! As the current time has been in round2, I would like to ask you whether this task is still open now? And where has it gone? I am very interested, please feel free to contact me at vaporshawnn@gmail.com!