supranational / blst

Multilingual BLS12-381 signature library
Apache License 2.0
458 stars 175 forks source link

Rust bindings: add `blst_miller_loop_n` #136

Closed huitseeker closed 1 year ago

huitseeker commented 1 year ago

The rust bindings include blst_miller_loop: https://github.com/supranational/blst/blob/master/bindings/rust/src/bindings.rs#L951-L957 which essentially maps to blst_miller_loop_n(_, _, 1): https://github.com/supranational/blst/blob/735d01997ea3d277ad1975ce33174918f3b62dc6/src/pairing.c#L405-L410

This makes multi-pairings unwieldy in rust. There is of course the (great!) construction of the Pairing engine, see: https://github.com/supranational/blst/blob/3af87cc1eaf78875718992335a58c0b0069c70e1/bindings/rust/src/lib.rs#L184 and: https://github.com/supranational/blst/blob/3af87cc1eaf78875718992335a58c0b0069c70e1/src/aggregate.c#L17-L36

... and in fact blst_pairing_raw_aggregate already allows hacking a workaround to the lack of a binding for the multi_miller_loop_n. But that engine is built for signature aggregation, and so is encumbered by parameters (e.g. the DST) that make sense in this context only.

In a proving context, multi-miller loops are quite useful, but the signing context and its parameters are not a part of the picture. I suspect the absence of the blst_miller_loop_n is what led to e.g. this sequential implementation: https://github.com/sifraitech/kzg/blob/2bcb0aca7a13ac905dbfca0c60ce3b609b15b539/blst-from-scratch/src/kzg_proofs.rs#L74-L90 .. which presumably makes the blst bench numbers less nice than what they could be.

TL;DR: it would be nice to have access to blst_miller_loop_n, as it would help with multi-miller loops in non-signing contexts.

dot-asm commented 1 year ago

TL;DR: it would be nice to have access to blst_miller_loop_n, as it would help with multi-miller loops in non-signing contexts.

In the presented context it would actually be more beneficial to execute the pair of blst_miller_loop-s in different threads instead of aiming for the loop_n. Maybe it would be appropriate to reformulate the query in favour of a more generalized multi-input interface that would assess the amount of available threads and pick a suitable combination of calls? I mean on a single-cpu system it would call loop_n, and on a multi-cpu one - execute them in parallel. Unless the amount of inputs is large enough to execute loop_n in multiple threads. How does it sound?

It also sounds like there is room for optimization of pair-wise conversions to affine coordinates... I mean a conversion costs you an expensive inversion, but if there are multiple inputs, the cost can be divided by amount of inputs. There are interfaces for same-type conversion, but one can just as well mix the types...

huitseeker commented 1 year ago

In the presented context it would actually be more beneficial to execute the pair of blst_miller_loop-s in different threads instead of aiming for the loop_n. Maybe it would be appropriate to reformulate the query in favour of a more generalized multi-input interface that would assess the amount of available threads and pick a suitable combination of calls?

Yes, you are correct, I was indeed conflating loop_n and parallelism as approaches towards solving this. In fact, if the Pairing engine you already have in aggregation was described in more general terms, and made its hash_or_encode and dst arguments optional, I would be happy with that too. Here's one example of me trying to test this out: https://github.com/MystenLabs/fastcrypto/blob/a3e1b36a57f5932a79250c79335834d50cc98fc1/fastcrypto-zkp/src/unit_tests/verifier_tests.rs#L87-L121 and using it in a proving context: https://github.com/MystenLabs/fastcrypto/blob/a3e1b36a57f5932a79250c79335834d50cc98fc1/fastcrypto-zkp/src/verifier.rs#L220-L226

dot-asm commented 1 year ago

Resolved in https://github.com/supranational/blst/commit/fceea3e1d74f22c2c0fff812a43245aa6bbd4e07.