feat!: napi-rs binding

[wemeetagain]

Roughly equivalent rewrite of the v0.2.x branch using napi-rs and the blst crate

Key differences:

• There are some additional optional parameters in verification functions for validating public keys and signatures
  • NOTE: it will be important when integrating into lodestar to ensure that the proper validation is done when required
• This is a thin wrapper around the blst crate
• Added an agregateWithRandomness function
  • this is the reason for using an in-tree blst repo, we require a small upstream change to blst

Results are promising. This is a comparison between 0.2.x branch and this branch (labeled - next)

Batch verify benchmark
================================================================
1 - BLS verification                                                  268.6676 ops/s       3722072 ns/op    512 runs
1 - BLS verification batch                                            242.9782 ops/s       4115595 ns/op    486 runs
1 - BLS verification - next                                           449.2125 ops/s       2226118 ns/op    512 runs
1 - BLS verification batch - next                                     299.0539 ops/s       3343879 ns/op    512 runs
2 - BLS verification                                                  123.3517 ops/s       8106901 ns/op    247 runs
2 - BLS verification batch                                            156.6904 ops/s       6382013 ns/op    314 runs
2 - BLS verification - next                                           230.4509 ops/s       4339320 ns/op    461 runs
2 - BLS verification batch - next                                     293.2217 ops/s       3410389 ns/op    512 runs
4 - BLS verification                                                  67.31199 ops/s   1.485619e+7 ns/op    135 runs
4 - BLS verification batch                                            106.2063 ops/s       9415639 ns/op    213 runs
4 - BLS verification - next                                           113.5575 ops/s       8806111 ns/op    228 runs
4 - BLS verification batch - next                                     230.7779 ops/s       4333171 ns/op    462 runs
8 - BLS verification                                                  69.81707 ops/s   1.432315e+7 ns/op    140 runs
8 - BLS verification batch                                            128.1760 ops/s       7801772 ns/op    257 runs
8 - BLS verification - next                                           57.15242 ops/s   1.749707e+7 ns/op    115 runs
8 - BLS verification batch - next                                     169.8414 ops/s       5887845 ns/op    340 runs
16 - BLS verification                                                 15.27253 ops/s   6.547705e+7 ns/op     31 runs
16 - BLS verification batch                                           29.38258 ops/s   3.403377e+7 ns/op     59 runs
16 - BLS verification - next                                          28.05916 ops/s   3.563899e+7 ns/op     57 runs
16 - BLS verification batch - next                                    121.2640 ops/s       8246470 ns/op    243 runs
32 - BLS verification                                                 7.644837 ops/s   1.308072e+8 ns/op     16 runs
32 - BLS verification batch                                           15.19085 ops/s   6.582910e+7 ns/op     31 runs
32 - BLS verification - next                                          13.72704 ops/s   7.284892e+7 ns/op     28 runs
32 - BLS verification batch - next                                    81.34062 ops/s   1.229398e+7 ns/op    163 runs
64 - BLS verification                                                 4.649212 ops/s   2.150902e+8 ns/op     10 runs
64 - BLS verification batch                                           7.742434 ops/s   1.291584e+8 ns/op     16 runs
64 - BLS verification - next                                          7.079833 ops/s   1.412463e+8 ns/op     15 runs
64 - BLS verification batch - next                                    54.05686 ops/s   1.849904e+7 ns/op    109 runs
128 - BLS verification                                                3.864711 ops/s   2.587516e+8 ns/op      8 runs
128 - BLS verification batch                                          9.040753 ops/s   1.106103e+8 ns/op     19 runs
128 - BLS verification - next                                         3.608887 ops/s   2.770937e+8 ns/op      8 runs
128 - BLS verification batch - next                                   34.58927 ops/s   2.891070e+7 ns/op     70 runs

[twoeths]

this is amazing ❤️ , would be great if you can add a Readme on how to setup environment, build and run tests @wemeetagain

[wemeetagain]

@twoeths

You will need https://rustup.rs/ installed. Also very recommended to have rust extensions in vscode.

This branch currently only adds code so that side-by-side benchmarks can be run, with the napi-rs version running alongside the existing 0.2.x branch.

All new code is in the next/ directory. Run all napi-rs/rust commands from this directory. This will all be moved to the root when we're ready. Bindings are done using https://napi.rs/. Feel free to review the docs there, or look at this branch as an example for how it can be used. All bindings code lives in next/src/lib.rs -- this is the file that should be more carefully reviewed.

To build the bindings, run yarn build. This will both (re)compile the rust code and (re)generate js and ts bindings files (index.js and index.d.ts). For rust-specific commands, you can use cargo. Some useful commands: cargo fmt to lint/format the code. cargo build for a debug build. cargo add to add additional rust dependencies. See the docs for more.

To run a benchmark, first build all the bindings (both napi-rs and 0.2.x), then run from the root: node -r ts-node/register benchmark/XXX.ts

[matthewkeil]

Adding memory test results here for posterity

Napi SecretKey - 109  b / instance - 2803528 allocated by 26000 instances - 5767168 rss allocated
Napi PublicKey - 108  b / instance - 2032424 allocated by 19000 instances - 933888 rss allocated
Napi Signature - 108  b / instance - 2889680 allocated by 27000 instances - 5095424 rss allocated

[wemeetagain]

this LGTM

@matthewkeil want a last review before I merge and release?

[wemeetagain]

~not sure why CI just started failing :(~

Edit: node 22.5.0 broke