We need a way to generate a public/private key pair

[truedat101]

Review: https://docs.nem.io/en/address-components/address-components-2#generating-a-key-pair

Also section 3.1 of https://www.nem.io/wp-content/themes/nem/files/NEM_techRef.pdf

This is perhaps the first really challenging bit of crypto to handle. I think we want a KeyPairGenerator class that handles the work rather than exposing the ugly stuff to the developer. See KeyPair struct for the format.

[truedat101]

See also: https://docs.nem.io/en/nem-sdk/private-key#6-1-create-private-keys

[truedat101]

@pedroalvesbatista I think the main task for you here is to outline the steps to generate the private key and public key. I think we have most of the information on what to implement via NEM technical ref PDF and the .js source code. If you can outline the design needed from algorithm steps, I will work on #11 to get the design of the class done.

[truedat101]

Then you can fill in implementation in the class using libsodium.

[truedat101]

@pedroalvesbatista Please write up algorithm to generate public and private key. I believe it's documented somewhere in the nem technical docs.

[pedroalvesbatista]

Working on it, I'm breaking out the algorithm and implementing.

[truedat101]

👾

[pedroalvesbatista]

randombytes_hash(hash.ptr, hash.length);

[pedroalvesbatista]

How to import and generate;

Put it into a variable called hash and do the transforms

[pedroalvesbatista]

source/keypair.d(30,9): Error: undefined identifier randombytes_hash source/keypair.d(39,16): Error: cannot implicitly convert expression hash of type ubyte[8] t o string dmd failed with exit

[truedat101]

Moving this further into proto3.

[pedroalvesbatista]

I guess for generating hash from the pkBytes ubyte[], we need to have some generalhash into crypto, right ?

[pedroalvesbatista]

Also, is better to work from ubyte and then cast to string, or generate the hash directly in strings ?

[truedat101]

So what is left to do on this? It's marked at 50% complete.

[pedroalvesbatista]

Here we have to find out convert the generated pkBytes into a hash

[truedat101]

@pedroalvesbatista where is the documentation of the algorithm steps to generate : public key, private key? I thought you said you documented this information. Please put into description.

[pedroalvesbatista]

@truedat101 The algorithm description could be better documented into the wiki, right ?

[truedat101]

Sure wiki is better

[pedroalvesbatista]

source/keypair.d(64,27): Error: function deimos.sodium.crypto_generichash.cryptogenerichash (ubyte* out, ulong outlen, const(ubyte) in_ , ulong inlen, const(ubyte) key, ulong keylen) is not callable using argument types (ubyte[][32], ulong, typeof(null), int) source/keypair.d(66,16): Error: cannot implicitly convert expression generalHash of type ubyte[][32] to ubyte[] dmd failed with exit code 1.

[pedroalvesbatista]

I want to make sure general hash is working properly, so, I'm getting this error. I know it's trivial about the types that is in use here, just a light in the path would be helpful \o/

[truedat101]

I can’t look at the source right now but remember we have to use the c interfaces of D to work with the C binding of sodium. That means referencing the key not as an a d array but as a pointer to the array.

[truedat101]

Something like buf.ptr, buf.length

[pedroalvesbatista]

Right, I'll test in double-way with hexData and mix all, see the final result and jump straight to the final algorithm.

[pedroalvesbatista]

I've tried the hexString! from std.conv to see if the pkBytes can be converted to hexadecimal string and then use genericHash to hash it

[truedat101]

Note: b47a5ee is meant to state "hex conversion functions" in the comment, not hash.

[truedat101]

@pedroalvesbatista most of what you've done here is checking in little idea sketches, but mostly non-compiling, non-functional code.

We need to finish this. It's been nearly two months on just this one item. I think the approach needs to change.

1. Don't code and drive. In other words, don't be reading a map and trying to drive. It's very difficult to code and arrive at the right place when you don't know where it is you try to go. You don't seem to be knowing exactly what the end goal is, so it's hard to ever get there. The end goal is a public key and a private key generated according to the NEM spec. You need to write up the complete algorithm needed to be implemented in order to generate the public key and the private key. I've been asking for that in the issue, but you haven't written up.
2. Write up your spec. You put up a stub here but never filled it in: https://github.com/IoTone/TheShellProject/wiki/Algorithm. Fill in the public key section and the private key section.
3. Make sure you end stand each step in what you've written up. We have 100% of the source code implementation in JS to review, any questions can be asked on this issue here #10 .
4. Start coding each step as a proper function. Stubs are ok, but please stop checking in stuff that is obviously broken. It's completely ok when you work by yourself, but if you work on a team, you can't do that. That means: valid and useful return types. When you don't know what it should be, auto is ok temporarily. Use valid parameters. How is it supposed to work that you have no argument functions that will return a stringToHex conversion without any arguments? Use parameters. If you are stuck because of some error , please post your code and ask questions. You can post a gist if you are stuck, for example. Use run.dlang.io to share code snippets.
5. Fill in functionality. Again, a lot of what is being checked in looks like sketches of something, but not looking like anything close to what is needed. If you don't know the proper syntax, don't check in imaginary Dlang syntax. Don't make up syntax. If you don't know the proper syntax, just check in commented-out pseudo code.
6. Add unittest() blocks to unit test your each function. The nice thing is that this is (1) easy to do, and (2) easy to copy how D's samples show how they unit test something easily: https://dlang.org/phobos/std_string.html#.representation . See the example code there. It's very short, one line assertion that validates some assumption about expected behavior.

Please restart your effort on this issue. I want to wrap this up by Friday. Focus on the algorithm of NEM for public and private key, get that written down what we need to implement. Don't worry as much about not understanding D fully yet. I can implement quickly once I see what is needed or missing. It takes me about 1/10th of the time to do this. Also, you have Ali if you have D specific questions.

[truedat101]

Also, this seems very far from 75% complete.

[pedroalvesbatista]

Well, acording to the docs, we must use

define MESSAGE ((const unsigned char *) "Arbitrary data to hash")

define MESSAGE_LEN 22

unsigned char hash[crypto_generichash_BYTES]; crypto_generichash(hash, sizeof hash, MESSAGE, MESSAGE_LEN, NULL, 0);

to give the general hash attribute. As D doesn't have unsigned char, I guess we can use ubyte[] for this, the MESSAGE and MESSAGE_LEN I can suppose as being the data we want to hash and it's length (maybe this can be hash.length).

What is being missed here to make generic hash work properly ?

[pedroalvesbatista]

Would make sents to use a single-part key in this hashing way ?

[pedroalvesbatista]

source/keypair.d(92,27): Error: function deimos.sodium.crypto_generichash.cryptogenerichash (ubyte* out, ulong outlen, const(ubyte) in_ , ulong inlen, const(ubyte) key, ulong keylen) is not callable using argument types (ubyte[]*, ulong) source/keypair.d(96,16): Error: cannot implicitly convert expression hashedData of type ubyte[][32] to ubyte[] dmd failed with exit code 1.

I'm stuck in this output. Maybe is the length type of what I'm using.

[pedroalvesbatista]

Any hint on how to proceed ?

[pedroalvesbatista]

Should this problem be something related to syntax or how I'm working with the ubyte[] type ?

[truedat101]

Can you post the line/method code that you are calling so I can better tell the problem?

[pedroalvesbatista]

ubyte[] generateHash(ubyte[] data, ubyte[] datain) {

    ubyte[] dataLength;

    ubyte[8] buf;

    ubyte[] [crypto_generichash_BYTES] hash;

    crypto_generichash(hash, hash.sizeof, data, datain, null, 0);

    auto hashedData = hash;

    return hashedData;
}

That's the code part that I try to call the generic hash.

[pedroalvesbatista]

I got this output when running the binary ./shell-d :

The local address is :
Hash of actual address :
std.conv.ConvException@/usr/include/dmd/phobos/std/conv.d(3530): Can't parse string: "[" is missing
----------------
??:? pure @safe void std.conv.parseCheck!(std.conv.parse!(ubyte[], immutable(char)[]).parse(ref immutable(char)[], dchar, dchar, dchar).s)
.parseCheck(dchar, immutable(char)[], ulong) [0xbddf74a9]
??:? pure @safe ubyte[] std.conv.parse!(ubyte[], immutable(char)[]).parse(ref immutable(char)[], dchar, dchar, dchar) [0xbddf71f1]
??:? pure @safe ubyte[] std.conv.toImpl!(ubyte[], immutable(char)[]).toImpl(immutable(char)[]) [0xbddf8233]
??:? pure @safe ubyte[] std.conv.to!(ubyte[]).to!(immutable(char)[]).to(immutable(char)[]) [0xbddf71af]
??:? _Dmain [0xbde01048]

[truedat101]

This looks like a bug in ??? Who is the caller. Somewhere in our code?

[pedroalvesbatista]

I'm looking into this to see if is something in the code.

[pedroalvesbatista]

Seems to be the std.conv when it allocates the immutable in memory and convert to the hexdecimal reference onto the memory reference address, or not ?

[truedat101]

You are missing the critical line in your error:

0x00007FF68F310B7A in D main at C:\Users\DavidJ\dev\projects\TheShellProject.git\shell-d\prototype\source\app.d(32)

[pedroalvesbatista]

Oh, how could I miss this ?

Well, so far so good, it's running. I'll post the output, I think I would expect some hexdecimal stuff, but maybe this is the norm of how this is done instead.

[pedroalvesbatista]

Entry point for The Shell Blockchain.
Starting to call block and address managers...
Your public keypair is: [0, 0, 0, 0, 0, 0, 0, 0]
Hashed public key: [18, 196, 234, 118, 73, 176, 32, 17, 178, 207, 190, 221, 249, 140, 118, 107, 149, 127, 143, 215, 146, 156, 118, 19, 66,
 227, 98, 133, 82, 42, 131, 104]

[pedroalvesbatista]

Maybe should we use the generic hash with single-key or not so for now ?

[pedroalvesbatista]

Also, there the crypto_box_keypair for generating keypairs is worth to use here ? Some parts concerning the keypairs for private keys could be good. Now we're about to explore the foremost hard parts from libsodium for a chain of crypto ops.

[pedroalvesbatista]

Perhaps now is the time to use all the methods that you had included which takes any value and converts to hex strings, right ? This will be the real output for keys. I'll take a look in how to do this the right way in D while I write all necessary code to start persisting it.

[truedat101]

From the above output: public key pair looks wrong. Also, probably want to output as hexencoded or as unencoded string, as the byte output isn't useful.

[pedroalvesbatista]

I'm trying to understand what's happening, so the public key pair looked odd. I will persist all that just to test the persistence and jump into address and account, for testing purposes, and when key pair show up as expected that will reflect into accounts and address. For now I see that whatever is being generated can satisfy some conditions for addresses and accounts.

[truedat101]

But use the DDB persistence thing directly, since I don't think we actually implemented anything useful in persistence yet. But anyway, don't complicate the keypair generation with persistence yet. just get it generating first.

[pedroalvesbatista]

Alright. I'll use the DDDB as well, as it is in persistence module, just to testing. The key pair with zero values is the question now on why is this happening. I'm breaking through it.

[truedat101]

Ok, so at this point, it's really about wrapping up the key pair generation. Let's make a note of what's left. @pedroalvesbatista I think you can refer back to the JS. We've generated a hash, and we can generate the random number. What's next?

[pedroalvesbatista]

The algorithm specify transforming anything else to hexstring, I guess you had implemented that, will look after and test to see how this is working.

[pedroalvesbatista]

What should be the ua2words method ? I guess this is CryptoJS specific. I don't have any notice on how this works, but looking into it.