dtr-org / unit-e

A digital currency for a new era of decentralized trust
https://unit-e.io
MIT License
45 stars 15 forks source link

Optimize bytes-related functions in messages.py #832

Closed castarco closed 5 years ago

castarco commented 5 years ago

This PR applies some minor changes that improve performance for some functions that manipulate byte arrays.

For the function sha256, we access the shalib.256 object directly, which saves some indirections (and it looks nicer too):

In [2]: def sha256_old(s):
   ...:     return hashlib.new('sha256', s).digest()
   ...: 

In [3]: def sha256_new(s):
   ...:     return hashlib.sha256(s).digest()
   ...:

In [4]: %timeit sha256_old(phrase)
The slowest run took 7.76 times longer than the fastest. This could mean that an intermediate result is being cached.
1000000 loops, best of 3: 968 ns per loop

In [5]: %timeit sha256_new(phrase)
The slowest run took 9.52 times longer than the fastest. This could mean that an intermediate result is being cached.
1000000 loops, best of 3: 620 ns per loop

For ser_uint256, we just rely on some built-in methods from the Python's int type:

In [6]: def ser_uint256_old(u):
   ...:     rs = b""
   ...:     for i in range(8):
   ...:         rs += struct.pack("<I", u & 0xFFFFFFFF)
   ...:         u >>= 32
   ...:     return rs
   ...: 

In [7]: def ser_uint256_new(u):
   ...:     return int(u).to_bytes(32, 'little')
   ...:

In [8]: %timeit ser_uint256_old(105752752996721010526070019734402373604975086831773275823333741804099920678329)
The slowest run took 9.45 times longer than the fastest. This could mean that an intermediate result is being cached.
100000 loops, best of 3: 2.65 µs per loop

In [9]: %timeit ser_uint256_new(105752752996721010526070019734402373604975086831773275823333741804099920678329)
The slowest run took 4.02 times longer than the fastest. This could mean that an intermediate result is being cached.
1000000 loops, best of 3: 374 ns per loop

For uint256_from_str we rely again on some int related methods:

In [10]: def uint256_from_str_old(s):
    ...:     r = 0
    ...:     t = struct.unpack("<IIIIIIII", s[:32])
    ...:     for i in range(8):
    ...:         r += t[i] << (i * 32)
    ...:     return r
    ...: 

In [11]: def uint256_from_str_new(s):
    ...:     return int.from_bytes(s[:32], 'little')
    ...: 

In [12]: hashed_phrase = hashlib(b'hello world').digest()

In [13]: %timeit uint256_from_str_old(hashed_phrase)
The slowest run took 13.18 times longer than the fastest. This could mean that an intermediate result is being cached.
1000000 loops, best of 3: 1.55 µs per loop

In [14]: %timeit uint256_from_str_new(hashed_phrase)
The slowest run took 6.61 times longer than the fastest. This could mean that an intermediate result is being cached.
1000000 loops, best of 3: 274 ns per loop

Signed-off-by: Andres Correa Casablanca andres@thirdhash.com

Gnappuraz commented 5 years ago

Maybe this could be back-ported to Bitcoin?

castarco commented 5 years ago

Maybe this could be back-ported to Bitcoin?

Yep, I think so. I'll try to send a patch.