[Harmony] BLS incompatibility issue after fork update

[gupadhyaya]

@herumi first of all, thank you for your contribution to this library. like many other projects, Harmony has been utilizing your library for block signing using bls. I am a developer at harmony and I have a question regarding upgrading to the latest bls standard. we originally used BLS_SWAP_G=1 with G1 public keys and G2 signatures with default generators (base points) as shown below:

x = 1490485673189267324327220854985940498515857427639219520252677586669310050426096250529683483057578053845695977302605
y = 1405772507307650904885814159424737301884803741041201599376687588686566229941847930862592328097124042659031385407926

the latest updates to your bls and mcl libraries needed us to change to BLS_ETH flag, however all our previous secret keys now are not compatible. possibly due to the changed generators (base points), as per eth2 bls standards:

x = 3685416753713387016781088315183077757961620795782546409894578378688607592378376318836054947676345821548104185464507
y = 1339506544944476473020471379941921221584933875938349620426543736416511423956333506472724655353366534992391756441569

my questions are:

• is there a way for us to automatically become compatible without having to redo all the secret keys & corresponding public keys (based on different base point) or it is just not possible?
• what are the configurations in the updated bls & mcl corresponding to the original setup of ours?

looking forward to your reply. thanks

[herumi]

Oh!, I don't think there are some users with BLS_SWAP_G and !BLS_ETH. Is it okay that I add a setBasePoint function to use the old generator?

[gupadhyaya]

Oh!, I don't think there are some users with BLS_SWAP_G and !BLS_ETH. Is it okay that I add a setBasePoint function to use the old generator? @herumi 1) so, we won't be able to reuse the secret keys with standard base points? 2) will setBasePoint be sufficient? before, the base points were initialized via mapToG1(&b, g_P, 1)

[herumi]

so, we won't be able to reuse the secret keys with standard base points?

If you set the old basepoint, then you don't have to change a pair of secretKey and publicKey.

will setBasePoint be sufficient? before, the base points were initialized via

It is sufficient to call blsSetGeneratorOfPublicKey once after blsInit. If you use the same signature then call mclBn_setMapToMode(MCL_MAP_TO_MODE_ORIGINAL) once.

[herumi]

@gupadhyaya Have you resolved this issue?

[rlan35]

@gupadhyaya Have you resolved this issue?

@herumi Hi, Herumi, since we launched our mainnet with herumi lib back in June 2019, we are inheriting the old code that's not eth2.0 compatible. Now we want to upgrade to blst for the performance and compatibility purpose. But we don't want to involve two libraries in our binary at the same time, and we want to configure blst to be compatible with the old herumi code so the old blocks is verifiable.

Can you let us know what's the changes that you made to make the old herumi lib compatible with eth 2.0? So that we can reverse engineer the blst lib to be compatible with the old logic in herumi lib that's in our mainnet already.

Thanks

[rlan35]

Actually we are hitting some obstacles in changing blst to be compatible with the old herumi code. Here is there thread with blst team: https://github.com/supranational/blst/issues/87 @herumi please help us.

[herumi]

Do you use https://github.com/harmony-one/bls and https://github.com/harmony-one/mcl ?

[herumi]

And do you compile them with BLS_SWAP_G=1?

[rlan35]

And do you compile them with BLS_SWAP_G=1?

Yes, we used the swapped curves initially trying to be compatible with eth 2.0 at that moment.

Do you use https://github.com/harmony-one/bls and https://github.com/harmony-one/mcl ?

Yes, that's the lib we are using.

[herumi]

I made a test repository to compare the old and new libraries.

This configuration seems the same result as the old library. https://github.com/herumi/test-harmony-bls/blob/main/harmony_test.cpp#L126-L141 Is this okay?

[herumi]

Now we want to upgrade to blst for the performance and compatibility purpose.

The pairing speed of the latest mcl/bls library is a little faster than blst on Core i7 and supports Eth 2.0 specification (compiled with BLS_ETH=1). And the library supports the old parameters as above.

[rlan35]

Now we want to upgrade to blst for the performance and compatibility purpose.

The pairing speed of the latest mcl/bls library is a little faster than blst on Core i7 and supports Eth 2.0 specification (compiled with BLS_ETH=1). And the library supports the old parameters as above.

That's good to hear. Did you have the benchmark result for it?

[rlan35]

I made a test repository to compare the old and new libraries.

This configuration seems the same result as the old library. https://github.com/herumi/test-harmony-bls/blob/main/harmony_test.cpp#L126-L141 Is this okay?

So let's assume the latest mcl/bls library is faster than bls, how do you recommend us to proceed to take advantage of it? Can we just use the latest mcl/bls lib as is. Or this https://github.com/herumi/test-harmony-bls/blob/main/harmony_test.cpp#L126-L141 is what we need to add?

It will be nice if there is a instruction for the users who need the compatibility with old mcl/bls lib.

[herumi]

Did you have the benchmark result for it?

I tested only pairing benchmark on Core i7-8700 ad Xeon Platinum 8280 with turbo-boost off. Could you please measure it at yourself if necessary?

The latest mcl/bls with -DBLS_ETH=1 provides the APIs compatible with Ethereum 2.0. (Remark : it does not use signHash and verifyHash.)

After blsInit(MCL_BLS12_381, MCLBN_COMPILED_TIME_VAR);, the setup

blsSetETHserialization(0);
blsSetMapToMode(0);
blsSetETHmode(0);
{
  const char *genStr = "1 4f58f3d9ee829f9a853f80b0e32c2981be883a537f0c21ad4af17be22e6e9959915ec21b7f9d8cc4c7315f31f3600e5 1212110eb10dbc575bccc44dcd77400f38282c4728b5efac69c0b4c9011bd27b8ed608acd81f027039216a291ac636a8";
  blsPublicKey gen;
  ret = blsPublicKeySetHexStr(&gen, genStr, strlen(genStr));
  if (ret != 0) {
    return 1;
  }
  ret = blsSetGeneratorOfPublicKey(&gen);
  if (ret != 0) {
    return 1;
  }
}

provides the old APIs of mcl/bls compatible with harmony.

[herumi]

Here is a sample code: https://github.com/herumi/bls-eth-go-binary/blob/master/bls/harmony_test.go .