"Ethereum" Serialization/deserialization

[vincenzoiovino]

Hello, Could you kindly explain or give a pointer to what the Ethereum serialization/deserialization is supposed to implement, especially for GT elements? Do you mean that there is some standardized way you follow for serializing/deserializing elements of GT? Anyway how is the this done for GT?

To your knowledge are there other libraries for the BLS12 pairing that are compatible with your way of serializing/deserializing GT and in general also G1 and G2? Thanks in advance. Kind regards

[herumi]

The serialization of G1/G2 of my library is standards-compliant. But I don't know whether GT serialization is standardized. GT is a subgroup of Fp12. GT is defined at notation. The serialization of Fp is bytes in big-endian. My library serializes the extended fields as follows. a+bi in Fp2 where a and b are in Fp. The serialization of Fp2 is the concatenation of serialization of a and b. a + bv + cv in Fp6 where a, b, and c are in Fp2. The serialization of Fp6 is the concatenation of serialization of a, b, and c. a + bw in Fp12 where a and b are in Fp6. The serialization of Fp2 is the concatenation of serialization of a and b.

[vincenzoiovino]

Thanks. Can I have a pointer to the documents that define the Ethereum serialization of G1/G2? Or an explanation here is also sufficient. Moreover, beyond serialization/deserialization is there some check done on membership and which one for G1/G2/GT? Thanks in advance

[herumi]

See https://github.com/herumi/mcl/blob/master/api.md#faq .

beyond serialization/deserialization is there some check done on membership and which one for G1/G2/GT?

The deserialization of G1/G2 always checks whether it is a valid point or not. The deserialization of GT does not check it because it is always valid as Fp12.

[vincenzoiovino]

It is always valid as FP12 but might it not be the case that it does not fall in the right subgroup? I mean it can be an element of FP12 but not of GT. Am I wrong? For the same reason you can have a valid EC point that is not in the right subgroup G1

[herumi]

I mean it can be an element of FP12 but not of GT.

Yes. For x in Fp12, x^r = 1 <=> x in GT. Do you need mclBnT_isValid()?

For the same reason you can have a valid EC point that is not in the right subgroup G1

The deserializer of G1/G2 checks whether the order of the point is correct.

[vincenzoiovino]

Owner

Yes, so does deserialization/serialization check validity or not?

[herumi]

It is very heavy computation to compute x^r for x in Fp12. It is heavier than calculation one pairing. So I will not modify the deserialization of GT (for backward compatibility). Could you call mclBnGT_isValid by yourself if necessary?

[vincenzoiovino]

Ok this was just to know. So I can assume that membership tests are not done for GT. Thanks for all (btw, this is out of scope but my tests showed that GT exponentiation was instead faster than pairings on your library and also others.)

[herumi]

I added mcl::bn::isValidGT(). https://github.com/herumi/mcl/commit/fd122d59168fee5dfc96462800aeec4a2e4b1e04

[vincenzoiovino]

Thanks. We can close the issue

[vincenzoiovino]

Hello, sorry to reopen the issue. But I wanted to point out this: https://github.com/arkworks-rs/algebra/issues/658 The serialization of GT of mcl appears to be different from the one of arkworks for the curve bls12-381. In the thread someone claims that the the tower of extensions fields used to define the field in which GT is embedded is the same both in mcl and arkworks. Then, do you have any clue on why the serialization in the two libraries is different? Btw, I do not quite understand some comments in the thread but if I am not wrong they plan to change the serialization. I wish that if any change in serialization is done in mcl in the future, you will still allow to access previous serialization/deserialization routines for backwards compatibility.

[vincenzoiovino]

update: I thought that maybe they use different representations for field elements, which one is used in mcl?

[herumi]

If the size of serialized data of an element in GT by another library is 48*12 bytes, then we need to know the order of Fp.

// t.cpp
#include <mcl/bls12_381.hpp>
#include <fstream>

using namespace mcl::bn;

int main()
{
    initPairing(mcl::BLS12_381);
    Fp::setETHserialization(true);
    GT x;
    Fp *p = x.getFp0();
    for (int i = 0; i < 12; i++) {
        p[i] = i + 1;
    }
    printf("x=%s\n", x.getStr().c_str());
    std::ofstream ofs("x.dat", std::ios::binary);
    x.save(ofs);
}

g++ t.cpp lib/libmcl.a -I include/ && ./a.out

I think we can find the order to deserialize the x.dat by another library. Or serialize an element of GT by another library and save it into "x.dat", exec the code:

Fpp12 x;
std::ifstream ifs("x.dat");
x.load(ifs);
printf("%s\n", x.getStr().c_str());

[herumi]

I thought that maybe they use different representations for field elements, which one is used in mcl?

See https://github.com/herumi/mcl/issues/182#issuecomment-1577925956 .

[herumi]

Select an element P in G1 and Q in G2, compute e = pairing(P, Q), and put e by mcl and another lib. Then by comparing them and we will see the difference.

[vincenzoiovino]

I pair the G1 generator with the G2 generator ( I already checked and the generators in both libraries are the same and they are serialized in that same way). This is the output in MCL of the pairing of these two elements: 089a1c5b46e5110b86750ec6a532348868a84045483c92b7af5af689452eafabf1a8943e50439f1d59882a98eaa0170f1250ebd871fc0a92a7b2d83168d0d727272d441befa15c503dd8e90ce98db3e7b6d194f60839c508a84305aaca1789b6193502b86edb8857c273fa075a50512937e0794e1e65a7617c90d8bd66065b1fffe51d7a579973b1315021ec3c19934f1368bb445c7c2d209703f239689ce34c0378a68e72a6b3b216da0e22a5031b54ddff57309396b38c881c4c849ec23e87018107154f25a764bd3c79937a45b84546da634b8f6be14a8061e55cceba478b23f7dacaa35c8ca78beae9624045b4b601b2f522473d171391125ba84dc4007cfbf2f8da752f7c74185203fcca589ac719c34dffbbaad8431dad1c1fb597aaa506fba23eb7c5af0d9f80940ca771b6ffd5857baaf222eb95a7d2809d61bfe02e1bfd1b68ff02f0b8102ae1c2d5d5ab1a19f26337d205fb469cd6bd15c3d5a04dc88784fbb3d0b2dbdea54d43b2b73f2cbb12d58386a8703e0f948226e47ee89d03350f55a7aefcd3c31b4fcb6ce5771cc6a0e9786ab5973320c806ad360829107ba810c5a09ffdd9be2291a0c25a99a211b8b424cd48bf38fcef68083b0b0ec5c81a93b330ee1a677d0d15ff7b984e8978ef48881e32fac91b93b47333e2ba570f41e58663bf08cf068672cbd01a7ec73baca4d72ca93544deff686bfd6df543d48eaa24afe47e1efde449383b67663104c581234d086a9902249b64728ffd21a189e87935a954051c7cdba7b3872629a4fafc05066245cb9108f0242d0fe3ef

The output in arkworks is: b68917caaa0543a808c53908f694d1b6e7b38de90ce9d83d505ca1ef1b442d2727d7d06831d8b2a7920afc71d8eb50120f17a0ea982a88591d9f43503e94a8f1abaf2e4589f65aafb7923c484540a868883432a5c60e75860b11e5465b1c9a08873ec29e844c1c888cb396933057ffdd541b03a5220eda16b2b3a6728ea678034ce39c6839f20397202d7c5c44bb68134f93193cec215031b17399577a1de5ff1f5b0666bdd8907c61a7651e4e79e0372951505a07fa73c25788db6eb8023519a5aa97b51f1cad1d43d8aabbff4dc319c79a58cafc035218747c2f75daf8f2fb7c00c44da85b129113173d4722f5b201b6b4454062e9ea8ba78c5ca3cadaf7238b47bace5ce561804ae16b8f4b63da4645b8457a93793cbd64a7254f150781019de87ee42682940f3e70a88683d512bb2c3fb7b2434da5dedbb2d0b3fb8487c84da0d5c315bdd69c46fb05d23763f2191aabd5d5c2e12a10b8f002ff681bfd1b2ee0bf619d80d2a795eb22f2aa7b85d5ffb671a70c94809f0dafc5b73ea2fb0657bae23373b4931bc9fa321e8848ef78894e987bff150d7d671aee30b3931ac8c50e0b3b0868effc38bf48cd24b4b811a2995ac2a09122bed9fd9fa0c510a87b10290836ad06c8203397b56a78e9a0c61c77e56ccb4f1bc3d3fcaea7550f3503efe30f2d24f00891cb45620605fcfaa4292687b3a7db7c1c0554a93579e889a121fd8f72649b2402996a084d2381c5043166673b3849e4fd1e7ee4af24aa8ed443f56dfd6b68ffde4435a92cd7a4ac3bc77e1ad0cb728606cf08bf6386e5410f This in turn comes from serializing this "PairingOutput": QuadExtField(CubicExtField(QuadExtField(2819105605953691245277803056322684086884703000473961065716485506033588504203831029066448642358042597501014294104502 + 1323968232986996742571315206151405965104242542339680722164220900812303524334628370163366153839984196298685227734799 u), QuadExtField(2987335049721312504428602988447616328830341722376962214011674875969052835043875658579425548512925634040144704192135 + 3879723582452552452538684314479081967502111497413076598816163759028842927668327542875108457755966417881797966271311 u), QuadExtField(261508182517997003171385743374653339186059518494239543139839025878870012614975302676296704930880982238308326681253 + 231488992246460459663813598342448669854473942105054381511346786719005883340876032043606739070883099647773793170614 u)) + CubicExtField(QuadExtField(3993582095516422658773669068931361134188738159766715576187490305611759126554796569868053818105850661142222948198557 + 1074773511698422344502264006159859710502164045911412750831641680783012525555872467108249271286757399121183508900634 u), QuadExtField(2727588299083545686739024317998512740561167011046940249988557419323068809019137624943703910267790601287073339193943 + 493643299814437640914745677854369670041080344349607504656543355799077485536288866009245028091988146107059514546594 u), QuadExtField(734401332196641441839439105942623141234148957972407782257355060229193854324927417865401895596108124443575283868655 + 2348330098288556420918672502923664952620152483128593484301759394583320358354186482723629999370241674973832318248497 u)) * u) I checked and the latter 12 elements when represented in hex in big endian and concatenated togeter form the same string of before (the one that starts in b689...).

The size of the serialized elements in both libraries is the same, that is 48 *12 bytes (of course the serializazed string is the double because the strings are in hex).

Also, replying to your previous comments, I would need also to deserialize mcl strings into arkworks, it should be universal, GT elements in different libraries (for example mcl and arkworks) should be compatible.

Thanks a lot

[vincenzoiovino]

Solved, it seems that the elements above are in different endianness and the u and v pairs are switched :-) Precisely, it is like this: in arkworks a GT element is serialized as 6 pairs: (u1,v1),..,(u6, v6). In mcl the same element is instead: (f(v1), f(u1)), ..., (f(v6), f(u6)), where the function f just changes the endiannes of the input :-) (notice that not just the endianness changes but also the order in each pair).