search

ChainSafe / blst-ts

Typescript wrapper for https://github.com/supranational/blst native bindings, a highly performant BLS12-381 signature library
Other
18 stars 13 forks source link

feat: add perf tests to compare new and old versions #112

Closed matthewkeil closed 8 months ago

matthewkeil commented 9 months ago

Perf test the new and old library to understand the differences.

Compares individual functions and also compares multithreading using workers/libuv.

Results:


  PublicKey
    ✓ PublicKey serialization- Napi                                        1479290 ops/s    676.0000 ns/op        -    1162780 runs  0.909 s
    ✓ PublicKey serialization - Swig                                       2227171 ops/s    449.0000 ns/op        -    1837977 runs   1.01 s
    ✓ PublicKey deserialize - Napi                                        71042.91 ops/s    14.07600 us/op        -      39190 runs  0.606 s
    ✓ PublicKey deserialize - Swig                                        62208.40 ops/s    16.07500 us/op        -      16683 runs  0.303 s
    ✓ PublicKey deserialize and validate - Napi - 1 keys                  18243.51 ops/s    54.81400 us/op        -       6293 runs  0.404 s
    ✓ PublicKey deserialize and validate - Swig - 1 keys                  18287.55 ops/s    54.68200 us/op        -       6347 runs  0.404 s
    ✓ PublicKey deserialize and validate - Napi - 100 keys                183.7902 ops/s    5.440987 ms/op        -         58 runs  0.860 s
    ✓ PublicKey deserialize and validate - Swig - 100 keys                185.7100 ops/s    5.384739 ms/op        -         58 runs  0.816 s
    ✓ PublicKey deserialize and validate - Napi - 10000 keys              1.833837 ops/s    545.3048 ms/op        -         12 runs   7.17 s
    ✓ PublicKey deserialize and validate - Swig - 10000 keys              1.848243 ops/s    541.0543 ms/op        -         12 runs   7.05 s

  SecretKey
    ✓ SecretKey.fromKeygen - Napi                                         626566.4 ops/s    1.596000 us/op        -     653083 runs   1.11 s
    ✓ SecretKey.fromKeygen - Swig                                          1149425 ops/s    870.0000 ns/op        -     388348 runs  0.404 s
    ✓ SecretKey serialization- Napi                                        1201923 ops/s    832.0000 ns/op        -     748148 runs  0.707 s
    ✓ SecretKey serialization - Swig                                       1663894 ops/s    601.0000 ns/op        -     718747 runs  0.505 s
    ✓ SecretKey deserialization - Napi                                    865800.9 ops/s    1.155000 us/op        -     294670 runs  0.606 s
    ✓ SecretKey deserialization - Swig                                     1579779 ops/s    633.0000 ns/op        -     423705 runs  0.606 s
    ✓ SecretKey.toPublicKey - Napi                                        11143.18 ops/s    89.74100 us/op        -       2359 runs  0.303 s
    ✓ SecretKey.toPublicKey - Swig                                        11255.18 ops/s    88.84800 us/op        -       3522 runs  0.404 s
    ✓ SecretKey.sign - Napi                                               3147.554 ops/s    317.7070 us/op        -        953 runs  0.622 s
    ✓ SecretKey.sign - SWig                                               3161.905 ops/s    316.2650 us/op        -        958 runs  0.620 s

  Signature
    ✓ Signature serialization- Napi                                        1068376 ops/s    936.0000 ns/op        -     484594 runs  0.505 s
    ✓ Signature serialization - Swig                                       1472754 ops/s    679.0000 ns/op        -     646972 runs  0.505 s
    ✓ Signature deserialize - Napi                                        36337.21 ops/s    27.52000 us/op        -      16688 runs  0.505 s
    ✓ Signature deserialize - Swig                                        33959.32 ops/s    29.44700 us/op        -       9008 runs  0.303 s
    ✓ Signatures deserialize and validate - Napi - 1 sets                 13383.48 ops/s    74.71900 us/op        -      13681 runs   1.11 s
    ✓ Signatures deserialize and validate - Swig - 1 sets                 13754.78 ops/s    72.70200 us/op        -       3142 runs  0.303 s
    ✓ Signatures deserialize and validate - Napi - 100 sets               137.1629 ops/s    7.290603 ms/op        -         29 runs  0.717 s
    ✓ Signatures deserialize and validate - Swig - 100 sets               137.7695 ops/s    7.258498 ms/op        -         29 runs  0.712 s
    ✓ Signatures deserialize and validate - Napi - 10000 sets             1.364249 ops/s    733.0039 ms/op        -         12 runs   9.54 s
    ✓ Signatures deserialize and validate - Swig - 10000 sets             1.377252 ops/s    726.0833 ms/op        -         12 runs   9.44 s

  functions
    aggregatePublicKeys
      ✓ aggregatePublicKeys - Napi - 1 sets                                 585137.5 ops/s    1.709000 us/op        -     194457 runs  0.404 s
      ✓ aggregatePublicKeys - Swig - 1 sets                                  1112347 ops/s    899.0000 ns/op        -     318451 runs  0.404 s
      ✓ aggregatePublicKeys - Napi - 8 sets                                 141083.5 ops/s    7.088000 us/op        -      65996 runs  0.505 s
      ✓ aggregatePublicKeys - Swig - 8 sets                                 179372.2 ops/s    5.575000 us/op        -      49587 runs  0.303 s
      ✓ aggregatePublicKeys - Napi - 32 sets                                39985.61 ops/s    25.00900 us/op        -      10584 runs  0.303 s
      ✓ aggregatePublicKeys - Swig - 32 sets                                47687.17 ops/s    20.97000 us/op        -      12348 runs  0.303 s
      ✓ aggregatePublicKeys - Napi - 128 sets                               10218.16 ops/s    97.86500 us/op        -       2987 runs  0.404 s
      ✓ aggregatePublicKeys - Swig - 128 sets                               12051.82 ops/s    82.97500 us/op        -       3521 runs  0.429 s
      ✓ aggregatePublicKeys - Napi - 256 sets                               5107.096 ops/s    195.8060 us/op        -       1491 runs  0.505 s
      ✓ aggregatePublicKeys - Swig - 256 sets                               6057.265 ops/s    165.0910 us/op        -       2348 runs  0.631 s
    aggregateSignatures
      ✓ aggregateSignatures - Napi - 1 sets                                 363108.2 ops/s    2.754000 us/op        -     129599 runs  0.404 s
      ✓ aggregateSignatures - Swig - 1 sets                                 534473.5 ops/s    1.871000 us/op        -     225231 runs  0.505 s
      ✓ aggregateSignatures - Napi - 8 sets                                 66299.81 ops/s    15.08300 us/op        -      18486 runs  0.303 s
      ✓ aggregateSignatures - Swig - 8 sets                                 76698.88 ops/s    13.03800 us/op        -      21392 runs  0.303 s
      ✓ aggregateSignatures - Napi - 32 sets                                17605.63 ops/s    56.80000 us/op        -       5988 runs  0.404 s
      ✓ aggregateSignatures - Swig - 32 sets                                19456.77 ops/s    51.39600 us/op        -       6701 runs  0.404 s
      ✓ aggregateSignatures - Napi - 128 sets                               4439.236 ops/s    225.2640 us/op        -       1325 runs  0.533 s
      ✓ aggregateSignatures - Swig - 128 sets                               4865.044 ops/s    205.5480 us/op        -        965 runs  0.411 s
      ✓ aggregateSignatures - Napi - 256 sets                               2226.785 ops/s    449.0780 us/op        -        666 runs  0.761 s
      ✓ aggregateSignatures - Swig - 256 sets                               2419.257 ops/s    413.3500 us/op        -        720 runs  0.724 s
    aggregateVerify
      ✓ aggregateVerify - Napi - 1 sets                                     1074.974 ops/s    930.2550 us/op        -        324 runs  0.808 s
      ✓ aggregateVerify - Swig - 1 sets                                     1059.717 ops/s    943.6480 us/op        -        321 runs  0.806 s
      ✓ aggregateVerify - Napi - 8 sets                                     291.7572 ops/s    3.427508 ms/op        -         90 runs  0.815 s
      ✓ aggregateVerify - Swig - 8 sets                                     280.7650 ops/s    3.561697 ms/op        -        486 runs   2.25 s
      ✓ aggregateVerify - Napi - 32 sets                                    82.32757 ops/s    12.14660 ms/op        -         37 runs  0.969 s
      ✓ aggregateVerify - Swig - 32 sets                                    82.35990 ops/s    12.14183 ms/op        -         28 runs  0.855 s
      ✓ aggregateVerify - Napi - 128 sets                                   20.62186 ops/s    48.49223 ms/op        -         32 runs   2.09 s
      ✓ aggregateVerify - Swig - 128 sets                                   21.16315 ops/s    47.25195 ms/op        -         13 runs   1.14 s
      ✓ aggregateVerify - Napi - 256 sets                                   10.58410 ops/s    94.48138 ms/op        -         13 runs   1.81 s
      ✓ aggregateVerify - Swig - 256 sets                                   10.63203 ops/s    94.05540 ms/op        -         13 runs   1.80 s
    verifyMultipleAggregateSignatures
      ✓ verifyMultipleAggregateSignatures - Napi - 1 sets                   1017.039 ops/s    983.2460 us/op        -        309 runs  0.805 s
      ✓ verifyMultipleAggregateSignatures - Swig - 1 sets                   952.9863 ops/s    1.049333 ms/op        -        290 runs  0.806 s
      ✓ verifyMultipleAggregateSignatures - Napi - 8 sets                   269.1226 ops/s    3.715778 ms/op        -         84 runs  0.814 s
      ✓ verifyMultipleAggregateSignatures - Swig - 8 sets                   229.2152 ops/s    4.362713 ms/op        -         72 runs  0.817 s
      ✓ verifyMultipleAggregateSignatures - Napi - 32 sets                  75.40198 ops/s    13.26225 ms/op        -         33 runs  0.940 s
      ✓ verifyMultipleAggregateSignatures - Swig - 32 sets                  63.53545 ops/s    15.73924 ms/op        -         22 runs  0.856 s
      ✓ verifyMultipleAggregateSignatures - Napi - 128 sets                 19.42884 ops/s    51.46987 ms/op        -         13 runs   1.19 s
      ✓ verifyMultipleAggregateSignatures - Swig - 128 sets                 16.16818 ops/s    61.84989 ms/op        -         13 runs   1.36 s
      ✓ verifyMultipleAggregateSignatures - Napi - 256 sets                 9.768284 ops/s    102.3721 ms/op        -         12 runs   1.74 s
      ✓ verifyMultipleAggregateSignatures - Swig - 256 sets                 8.115408 ops/s    123.2224 ms/op        -         12 runs   2.10 s
    verifyMultipleAggregateSignatures same message
      ✓ Same message - Napi - 1 sets                                        982.9227 ops/s    1.017374 ms/op        -        297 runs  0.807 s
      ✓ Same message - Swig - 1 sets                                        985.3828 ops/s    1.014834 ms/op        -        299 runs  0.809 s
      ✓ Same message - Napi - 8 sets                                        617.7953 ops/s    1.618659 ms/op        -        187 runs  0.813 s
      ✓ Same message - Swig - 8 sets                                        644.9878 ops/s    1.550417 ms/op        -        131 runs  0.710 s
      ✓ Same message - Napi - 32 sets                                       275.0797 ops/s    3.635310 ms/op        -        141 runs   1.03 s
      ✓ Same message - Swig - 32 sets                                       298.3720 ops/s    3.351521 ms/op        -         61 runs  0.724 s
      ✓ Same message - Napi - 128 sets                                      86.06384 ops/s    11.61928 ms/op        -         28 runs  0.867 s
      ✓ Same message - Swig - 128 sets                                      94.88700 ops/s    10.53885 ms/op        -         41 runs  0.979 s
      ✓ Same message - Napi - 256 sets                                      44.13315 ops/s    22.65871 ms/op        -         16 runs  0.932 s
      ✓ Same message - Swig - 256 sets                                      49.82175 ops/s    20.07156 ms/op        -         17 runs  0.899 s

  multithreading perf
    libuv
      ✓ libuv multithreading - napi                                        0.2702050 ops/s    3.700894  s/op        -         14 runs   55.5 s
    workers
      ✓ worker multithreading - swig                                       0.2181829 ops/s    4.583311  s/op        -         12 runs   59.5 s
{ libuvPoolSize: 9, workerPoolSize: 9 }
matthewkeil commented 9 months ago

After digging through the SWIG generated code I am pretty confident now that I can swap the shared_ptr for a unique_ptr safely. There are no smart pointers used at all by swig and the objects are all meant to be singletons that are referenced by every other instance in JS land. The GC moves objects so there is no deep copying happening which means there will be no leaks and not race conditions or other memory instability from the switch. It will also significantly increase the dereferencing operation during execution because there will be no atomic operations related to the shared_ptr. I will make the swap and see how it improves the performance

I tested this and it did not turn out faster. I will check to see if there is a memory savings using unique vs shared pointers though