ebellocchia / bip_utils

Generation of mnemonics, seeds, private/public keys and addresses for different types of cryptocurrencies
MIT License
306 stars 84 forks source link

how to add speed #21

Closed Noname400 closed 3 years ago

Noname400 commented 3 years ago

Have you tested with: PYPY NIMBA

Noname400 commented 3 years ago

speed was 1.5 times more with ecdsa[gmpy2]

ebellocchia commented 3 years ago

Hello, no I've never tried, but if you'd like to test them and post the results you're welcome of course.

Regards, Emanuele

Noname400 commented 3 years ago

OK. Rezuls send.

Noname400 commented 3 years ago

Ubuntu 18.04 cpu: 1 x 2.8 ГГц memory: 1024 Мб

(only ecsda) bip-utils (1.11.1) bitarray (1.9.2) ecdsa (0.17.0)

BIP44 1.8299272060394287 sec (round(2000)) BIP32 9.162187337875366 sec (round(2000)) BIP44 9.948112964630127 sec (round(10000)) BIP32 49.783501625061035 sec (round(10000))

(ecsda[gmpy]) bip-utils (1.11.1) bitarray (1.9.2) ecdsa (0.17.0) gmpy (1.17)

Traceback (most recent call last): File "test_bip_44.py", line 14, in bip_obj_acc = bip_obj_mst.Purpose().Coin().Account(0) File "/usr/local/lib/python3.6/dist-packages/bip_utils/bip/bip44.py", line 135, in Purpose return self._PurposeGeneric(self) File "/usr/local/lib/python3.6/dist-packages/bip_utils/bip/bip44_base.py", line 364, in _PurposeGeneric return cls(bip_obj.m_bip32.ChildKey(cls._GetPurpose()), bip_obj.m_coin_type) File "/usr/local/lib/python3.6/dist-packages/bip_utils/bip/bip32.py", line 236, in ChildKey return self.CkdPriv(index) if not self.m_is_public else self.CkdPub(index) File "/usr/local/lib/python3.6/dist-packages/bip_utils/bip/bip32.py", line 421, in __CkdPriv secret = ConvUtils.IntegerToBytes(new_key_int) File "/usr/local/lib/python3.6/dist-packages/bip_utils/utils/conversion.py", line 57, in IntegerToBytes return data_int.to_bytes((data_int.bit_length() + 7) // 8, endianness) AttributeError: 'mpz' object has no attribute 'to_bytes'

(ecsda[gmpy2]) bip-utils (1.11.1) bitarray (1.9.2) ecdsa (0.17.0) gmpy (1.17) gmpy2 (2.0.8)

Traceback (most recent call last): File "test_bip_44.py", line 14, in bip_obj_acc = bip_obj_mst.Purpose().Coin().Account(0) File "/usr/local/lib/python3.6/dist-packages/bip_utils/bip/bip44.py", line 135, in Purpose return self._PurposeGeneric(self) File "/usr/local/lib/python3.6/dist-packages/bip_utils/bip/bip44_base.py", line 364, in _PurposeGeneric return cls(bip_obj.m_bip32.ChildKey(cls._GetPurpose()), bip_obj.m_coin_type) File "/usr/local/lib/python3.6/dist-packages/bip_utils/bip/bip32.py", line 236, in ChildKey return self.CkdPriv(index) if not self.m_is_public else self.CkdPub(index) File "/usr/local/lib/python3.6/dist-packages/bip_utils/bip/bip32.py", line 421, in __CkdPriv secret = ConvUtils.IntegerToBytes(new_key_int) File "/usr/local/lib/python3.6/dist-packages/bip_utils/utils/conversion.py", line 57, in IntegerToBytes return data_int.to_bytes((data_int.bit_length() + 7) // 8, endianness) AttributeError: 'mpz' object has no attribute 'to_bytes'

Noname400 commented 3 years ago

(ecsda[gmpy2]) bip-utils (1.6.0) bitarray (1.9.2) ecdsa (0.17.0) gmpy (1.17) gmpy2 (2.1.0b5)

BIP44 1.096778154373169 sec (round(2000)) BIP32 4.9470813274383545 sec (round(2000)) BIP44 4.855431795120239 sec (round(10000)) BIP32 24.186437606811523 sec (round(10000))

(ecsda[gmpy]) bip-utils (1.6.0) bitarray (1.9.2) ecdsa (0.17.0) gmpy (1.17)

BIP44 0.9768166542053223 sec (round(2000)) BIP32 4.581654071807861 sec (round(2000)) BIP44 4.645848274230957 sec (round(10000)) BIP32 22.78937530517578 sec (round(10000))

(ecsda) bip-utils (1.6.0) bitarray (1.9.2) ecdsa (0.17.0)

BIP44 0.9107656478881836 sec (round(2000)) BIP32 4.2871153354644775 sec (round(2000)) BIP44 4.743731260299683 sec (round(10000)) BIP32 23.580677270889282 sec (round(10000))

ebellocchia commented 3 years ago

Hi, I added a check for mpz class in IntegerToBytes function, I tried and it should run now with the last commit

Noname400 commented 3 years ago

ok. thx. i check

Noname400 commented 3 years ago
from bip_utils import Bip32, Bip44, Bip44Coins, Bip44Changes, Bip49, Bip32Utils

ImportError: cannot import name 'Bip32' from 'bip_utils' (C:\Python39\lib\site-packages\bip_utils-2.0.0-py3.9.egg\bip_utils__init__.py)

Noname400 commented 3 years ago

Package Version


base58 2.1.0 bip-utils 2.0.0

ebellocchia commented 3 years ago

Hi, as you can read in the README, in version 2.0.0 there is no Bip32 but 3 different classes: Bip32Ed25519Slip, Bip32Nist256p1 and Bip32Secp256k1

Noname400 commented 3 years ago

ok. thx VERDICT: version 1.11.1 is twice as fast as version 2.0.0. without GMPY2

Noname400 commented 3 years ago

how to get an address from an uncompressed key ?

ebellocchia commented 3 years ago

The compressed/uncompressed public key is selected automatically by the address classes depending on the specific coin. For example:

uncompr_key = binascii.unhexlify(b"04c3d01cb07697dc5105013bea2e73a896b6019ec3c5ea2b97dba14ae4456439f4ec9654b17e30a8a5232078201ecf5cc702dfbb70266aecf16b1f81d85e6b9942")
# For P2PKH, the compressed key is selected internally
P2PKHAddr.EncodeKey(Secp256k1PublicKey(uncompr_key))
# For ETH, the uncompressed key is selected internally
EthAddr.EncodeKey(Secp256k1PublicKey(uncompr_key))
Noname400 commented 3 years ago

how to do it in version 1.11.1? version 2.0.0 is very slow

Noname400 commented 3 years ago

Traceback (most recent call last): File "c:\test\test\test_bip_44.py", line 51, in addr2 = P2PKH.ToAddress(bip44_puc) File "C:\Python39\lib\site-packages\bip_utils\addr\P2PKH_addr.py", line 50, in ToAddress raise ValueError("Public compressed key is required for P2PKH") ValueError: Public compressed key is required for P2PKH

ebellocchia commented 3 years ago

In the old version you should do it manually by using the ecdsa library, you can find an example in test_eth_addr.py (always referring to 1.11.1) where it converts from compressed to uncompressed key (you just have to do the opposite).

ebellocchia commented 3 years ago

Hi, I think I identified the problem of the "slowness" with respect to the old version. I'll try to fix it in these days.

Thanks

Noname400 commented 3 years ago

Thanks for your work. I also want to ask, is it possible to work with the indication of languages? mnemonic_lang = ['english', 'chinese_simplified', 'french', 'spanish'] to choose only the right ones

ebellocchia commented 3 years ago

So, the problem was that private/public keys were created from bytes each time. This was not a big issue for ed25519 but it was a disaster for ecdsa, since it is a pure python implementation. Now I'm directly passing around objects instead of bytes to avoid this and I noticed a huge speed improvement:

It's still a little bit slower than 1.11.1 (4s, it's understandable since the code is more complex) but it's not so slow like before.

Different languages in BIP39 are already supported, I don't get your question.

Noname400 commented 3 years ago

I don't understand how to select only a few languages

ebellocchia commented 3 years ago

You can select one language at time. You pass the language type to the constructor of the mnemonic generator and you can use it to generate as many mnemonics as you want:

mnemonic = []
# Some mnemonics in italian
mnemonic_gen_it = Bip39MnemonicGenerator(Bip39Languages.ITALIAN)
for i in range(0, 20):
    mnemonic.append(mnemonic_gen_it.FromWordsNumber(Bip39WordsNum.WORDS_NUM_12))
# Some mnemonics in french
mnemonic_gen_fr = Bip39MnemonicGenerator(Bip39Languages.FRENCH)
for i in range(0, 20):
    mnemonic.append(mnemonic_gen_fr.FromWordsNumber(Bip39WordsNum.WORDS_NUM_12))

Available languages:

Language Enum
English Bip39Languages.ENGLISH
Italian Bip39Languages.ITALIAN
French Bip39Languages.FRENCH
Spanish Bip39Languages.SPANISH
Portuguese Bip39Languages.PORTUGUESE
Czech Bip39Languages.CZECH
Chinese (simplified) Bip39Languages.CHINESE_SIMPLIFIED
Chinese (traditional) Bip39Languages.CHINESE_TRADITIONAL
Korean Bip39Languages.KOREAN

Please note that Japanese is not currently supported (it's the only missing one)

Noname400 commented 3 years ago

I understand this implementation. but it is not very comfortable.

Noname400 commented 3 years ago

so you have to use a more flexible system https://github.com/trezor/python-mnemonic

Noname400 commented 3 years ago

mnemonic_lang = ['english', 'chinese_simplified', 'chinese_traditional', 'french'] mnemo = Mnemonic(mem) mnemonic:str = mnemo.generate(words) seed_bytes:bytes = mnemo.to_seed(mnemonic, passphrase='') for mem in inf.mnemonic_lang: for num in range(20): inf.count_32 = inf.count_32 + 2 bip32_ctx = Bip32.FromSeedAndPath(seed_bytes, "m/0/" + str(num)) # m/0/0 bip32_h160_1 = CryptoUtils.Hash160(bip32_ctx.PublicKey().RawCompressed().ToBytes()).hex() bip32_h160_2 = CryptoUtils.Hash160(bip32_ctx.PublicKey().RawUncompressed().ToBytes()).hex() print('* address Compress - {}'.format(bip32_ctx.PublicKey().ToAddress()))

Noname400 commented 3 years ago

from mnemonic import Mnemonic

Noname400 commented 3 years ago

image

ebellocchia commented 3 years ago

I know that library but I don't like that code design, I always prefer to assign a single responsibility to a single class (so one class is the mnemonic generator, one class is the mnemonic validator, one class is the seed generator). That's how I like to code. Anyway, I don't understand what's the problem for what you want to do, you just have to iterate over the languages:

for lang in Bip39Languages:
    mnemonic = Bip39MnemonicGenerator(lang).FromWordsNumber(Bip39WordsNum.WORDS_NUM_12)
    seed = Bip39SeedGenerator(mnemonic, lang).Generate()
Noname400 commented 3 years ago

Sorry, I didn't mean to offend you. your code is great. what you do is impressive. As you can see in my code only 5 languages ​​are used using higher code i iterate over all languages

Noname400 commented 3 years ago

My English very bad. Sorry

Noname400 commented 3 years ago

for lang in Bip39Languages: mnemonic = Bip39MnemonicGenerator(lang).FromWordsNumber(Bip39WordsNum.WORDS_NUM_12) seed = Bip39SeedGenerator(mnemonic, lang).Generate()

All Lang. Need 5

ebellocchia commented 3 years ago

Don't worry, I wasn't offended at all. Ok I got the point, so you can create a list with the languages you need like this:

mnemonic_lang = [Bip39Languages.ENGLISH, Bip39Languages.CHINESE_SIMPLIFIED, Bip39Languages.CHINESE_TRADITIONAL, Bip39Languages.FRENCH]

for lang in mnemonic_lang:
    mnemonic = Bip39MnemonicGenerator(lang).FromWordsNumber(Bip39WordsNum.WORDS_NUM_12)
    seed = Bip39SeedGenerator(mnemonic, lang).Generate()
Noname400 commented 3 years ago

this is what you need, thanks

ninjainfosec commented 1 year ago

please help me out with this from bip_utils import Bip39EntropyBitLen, Bip39EntropyGenerator, Bip39MnemonicGenerator, Bip39WordsNum, Bip39Languages ImportError: cannot import name 'Bip39Languages' from 'bip_utils' (/usr/local/lib/python3.10/dist-packages/bip_utils/init.py)

ebellocchia commented 1 year ago

Bip39Languages is in init.py, so there is some problems with the installation or you are using an older version of the library, you should try reinstalling it