Improve idemix performance on HLF

hyperledger / fabric

Hyperledger Fabric is an enterprise-grade permissioned distributed ledger framework for developing solutions and applications. Its modular and versatile design satisfies a broad range of industry use cases. It offers a unique approach to consensus that enables performance at scale while preserving privacy.

https://wiki.hyperledger.org/display/fabric

Apache License 2.0

15.65k stars 8.81k forks source link

Improve idemix performance on HLF #3356

Open mffrench opened 2 years ago

mffrench commented 2 years ago

Hello,

with @adecaro we identified some improvments task to provide better performance on the overall HLF transaction workflow when using idemix. I open that issue to keep track on them and see when such tasks can be implemented.

@adecaro : can you help to define here the different idemix improvments to be reported on HLF stack (crypto lib update, marshaling issue, ... ?). Thank you :)

adecaro commented 2 years ago

This can be easily address if we enhance https://github.com/IBM/idemix. Currently, to instantiate the Idemix BCCSP, we need to specify a mathlib curve and translator. Instead, by making all Idemix algorithms taking in input also the issuer public key (IPK, for short), we can use the IPK tell which curve and translator to use. When importing the IPK into the BCCSP, we just need to recognise which curve that IPK is based upon, and we are done.

@mffrench , @ale-linux, @yacovm, what do you think?

yacovm commented 2 years ago

Doesn't this require a capability? Otherwise how will old validation code that activates the old idemix be able to parse the new idemix signed transactions?

ale-linux commented 2 years ago

The change you suggest @adecaro would require a capability switch as @yacovm points out. I'd like to better understand the exact type of enhancements we're talking about - is there a list somewhere?

mffrench commented 2 years ago

@ale-linux : during our perf tests using idemix signature we noticed TPS on ordering service was very very low. Still during our perfs tests, @adecaro did perf improvments on FSC/FTS by updating crypto lib and more...

Also seems to me this very slow idemix behavior on the HLF TX workflow has a very bad behavior. In our current usage of idemix, we do following :

SDK GO call 1 : Put data with IDEMIX sig
SDK GO call 1 : return OK
SDK GO call 2 : Get previously added data with ID
SDK GO call 2 : return NOK (?!)
Sleep some time (~5sec)
SDK GO call 3 : Get previously added data with ID
SDK GO call 3 : return OK

So here we have some mismatch between SDK call put with idemix and its actual write in the SoW. I was thinking maybe it could be linked with the bad idemix perf overall the HLF tx flow ?