search

3rdIteration / btcrecover

BTCRecover is an open source wallet password and seed recovery tool. For seed based recovery, this is primarily useful in situations where you have lost/forgotten parts of your mnemonic, or have made an error transcribing it. (So you are either seeing an empty wallet or gettign an error that your seed is invalid) For wallet password or passphrase recovery, it is primarily useful if you have a reasonable idea about what your password might be.
https://btcrecover.readthedocs.io/
GNU General Public License v2.0
551 stars 178 forks source link

Vega GPU Not Supported on Mac #380

Open vshuraeff opened 1 year ago

vshuraeff commented 1 year ago

tried to run under python 3.9.15, installed requirements-full.txt, tried pyopencl versions 2021.1.1; 2021.2.2; but 2022.2.3 is only version which installed without errors. so I trying to run openCL test and get this: 

> btcrecover git:(master) PYOPENCL_COMPILER_OUTPUT=1 python -m btcrecover.test.test_seeds -v OpenCL_Tests 
test_BIP39_BTC_OpenCL_Brute (__main__.TestRecoveryFromAddress) ... ERROR

Stdout:
Using the 'en' wordlist.
Assuming a 12 word mnemonic. (This can be overridden with --mnemonic-length)
OpenCL: Auto Selecting Best Platform

Stderr:
/Users/vshuraeff/.asdf/installs/python/3.9.15/lib/python3.9/site-packages/pyopencl/__init__.py:270: CompilerWarning: Built kernel retrieved from cache. Original from-source build had warnings:
Build on <pyopencl.Device 'AMD Radeon Pro Vega 48 Compute Engine' on 'Apple' at 0x1021d00> succeeded, but said:

<program source>:59:19: warning: no previous prototype for function 'SWAP'
    unsigned long SWAP (const unsigned long val)
                  ^
<program source>:130:5: warning: comparison of integers of different signs: 'int' and 'const unsigned int'
    def_printFromWord(__private, printFromWord, "")
    ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
<program source>:116:27: note: expanded from macro 'def_printFromWord'
        for (int j = 0; j < len_bytes; j++){    \
                        ~ ^ ~~~~~~~~~
<program source>:131:5: warning: comparison of integers of different signs: 'int' and 'const unsigned int'
    def_printFromWord(__global, printFromWord_glbl, "")
    ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
<program source>:116:27: note: expanded from macro 'def_printFromWord'
        for (int j = 0; j < len_bytes; j++){    \
                        ~ ^ ~~~~~~~~~
<program source>:132:5: warning: comparison of integers of different signs: 'int' and 'const unsigned int'
    def_printFromWord(__private, printFromWord_n, "\n")
    ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
<program source>:116:27: note: expanded from macro 'def_printFromWord'
        for (int j = 0; j < len_bytes; j++){    \
                        ~ ^ ~~~~~~~~~
<program source>:133:5: warning: comparison of integers of different signs: 'int' and 'const unsigned int'
    def_printFromWord(__global, printFromWord_glbl_n, "\n")
    ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
<program source>:116:27: note: expanded from macro 'def_printFromWord'
        for (int j = 0; j < len_bytes; j++){    \
                        ~ ^ ~~~~~~~~~
<program source>:394:1: warning: comparison of integers of different signs: 'int' and 'const unsigned int'
def_hash(hash_global, __global, __global, md_pad__global, printFromLong_glbl_n)
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
<program source>:330:35: note: expanded from macro 'def_hash'
    for (int block_i = 0; block_i < nBlocks; block_i++)     \
                          ~~~~~~~ ^ ~~~~~~~
<program source>:395:1: warning: comparison of integers of different signs: 'int' and 'const unsigned int'
def_hash(hash_private, __private, __private, md_pad__private, printFromLong_n)
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
<program source>:330:35: note: expanded from macro 'def_hash'
    for (int block_i = 0; block_i < nBlocks; block_i++)     \
                          ~~~~~~~ ^ ~~~~~~~
<program source>:396:1: warning: comparison of integers of different signs: 'int' and 'const unsigned int'
def_hash(hash_glbl_to_priv, __global, __private, md_pad__global, printFromLong_glbl_n)
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
<program source>:330:35: note: expanded from macro 'def_hash'
    for (int block_i = 0; block_i < nBlocks; block_i++)     \
                          ~~~~~~~ ^ ~~~~~~~
<program source>:397:1: warning: comparison of integers of different signs: 'int' and 'const unsigned int'
def_hash(hash_priv_to_glbl, __private, __global, md_pad__private, printFromLong_n)
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
<program source>:330:35: note: expanded from macro 'def_hash'
    for (int block_i = 0; block_i < nBlocks; block_i++)     \
                          ~~~~~~~ ^ ~~~~~~~
<program source>:458:27: warning: comparison of integers of different signs: 'int' and 'unsigned long'
        for (int j = 0; j < end; j++){
                        ~ ^ ~~~
<program source>:488:23: warning: comparison of integers of different signs: 'int' and 'unsigned long'
    for (int j = 0; j < m_len_word; j++){
                    ~ ^ ~~~~~~~~~~
<program source>:630:20: warning: initializing '__global unsigned long *' with an expression of type 'unsigned long const __global[4]' discards qualifiers
    __global word *pwdBuffer = pwdbuffer_arg[0].buffer;
                   ^           ~~~~~~~~~~~~~~~~~~~~~~~

  warn(text, CompilerWarning)
test_BIP39_Eth_OpenCL_Brute (__main__.TestRecoveryFromAddress) ... ERROR

Stdout:
Using the 'en' wordlist.
Assuming a 12 word mnemonic. (This can be overridden with --mnemonic-length)
OpenCL: Auto Selecting Best Platform
test_Electrum_OpenCL_Brute (__main__.TestRecoveryFromAddress) ... ERROR

Stdout:
Using the 'en' wordlist.
OpenCL: Auto Selecting Best Platform

======================================================================
ERROR: test_BIP39_BTC_OpenCL_Brute (__main__.TestRecoveryFromAddress)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/vshuraeff/work/crypto/btcrecover/btcrecover/test/test_seeds.py", line 115, in skip_or_test
    test_func(self)
  File "/Users/vshuraeff/work/crypto/btcrecover/btcrecover/test/test_seeds.py", line 1191, in test_BIP39_BTC_OpenCL_Brute
    self.assertEqual(btcrseed.WalletBIP39._return_verified_password_or_false_opencl(wallet,
  File "/Users/vshuraeff/work/crypto/btcrecover/btcrecover/btcrseed.py", line 1000, in _return_verified_password_or_false_opencl
    clResult = self.opencl_algo.cl_pbkdf2(self.opencl_context_pbkdf2_sha512[i], cleaned_mnemonic_ids_list,
  File "/Users/vshuraeff/work/crypto/btcrecover/lib/opencl_brute/opencl.py", line 705, in cl_pbkdf2
    result = concat(self.opencl_ctx.run(bufStructs, func, iter(passwordlist), salt))
  File "/Users/vshuraeff/work/crypto/btcrecover/lib/opencl_brute/opencl.py", line 524, in concat
    return [obj for lval in ll for obj in lval]
  File "/Users/vshuraeff/work/crypto/btcrecover/lib/opencl_brute/opencl.py", line 524, in <listcomp>
    return [obj for lval in ll for obj in lval]
  File "/Users/vshuraeff/work/crypto/btcrecover/lib/opencl_brute/opencl.py", line 263, in run
    func(self, pwdim, pass_g, salt_g, result_g)
  File "/Users/vshuraeff/work/crypto/btcrecover/lib/opencl_brute/opencl.py", line 702, in func
    prg.pbkdf2(s.queue, pwdim, None, pass_g, salt_g, result_g,
  File "/Users/vshuraeff/.asdf/installs/python/3.9.15/lib/python3.9/site-packages/pyopencl/__init__.py", line 895, in kernel_call
    return self._enqueue(self, queue, global_size, local_size, *args, **kwargs)
  File "<pyopencl invoker for 'pbkdf2'>", line 8, in enqueue_knl_pbkdf2
pyopencl._cl.LogicError: clEnqueueNDRangeKernel failed: INVALID_PROGRAM_EXECUTABLE

Stdout:
Using the 'en' wordlist.
Assuming a 12 word mnemonic. (This can be overridden with --mnemonic-length)
OpenCL: Auto Selecting Best Platform

Stderr:
/Users/vshuraeff/.asdf/installs/python/3.9.15/lib/python3.9/site-packages/pyopencl/__init__.py:270: CompilerWarning: Built kernel retrieved from cache. Original from-source build had warnings:
Build on <pyopencl.Device 'AMD Radeon Pro Vega 48 Compute Engine' on 'Apple' at 0x1021d00> succeeded, but said:

<program source>:59:19: warning: no previous prototype for function 'SWAP'
    unsigned long SWAP (const unsigned long val)
                  ^
<program source>:130:5: warning: comparison of integers of different signs: 'int' and 'const unsigned int'
    def_printFromWord(__private, printFromWord, "")
    ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
<program source>:116:27: note: expanded from macro 'def_printFromWord'
        for (int j = 0; j < len_bytes; j++){    \
                        ~ ^ ~~~~~~~~~
<program source>:131:5: warning: comparison of integers of different signs: 'int' and 'const unsigned int'
    def_printFromWord(__global, printFromWord_glbl, "")
    ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
<program source>:116:27: note: expanded from macro 'def_printFromWord'
        for (int j = 0; j < len_bytes; j++){    \
                        ~ ^ ~~~~~~~~~
<program source>:132:5: warning: comparison of integers of different signs: 'int' and 'const unsigned int'
    def_printFromWord(__private, printFromWord_n, "\n")
    ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
<program source>:116:27: note: expanded from macro 'def_printFromWord'
        for (int j = 0; j < len_bytes; j++){    \
                        ~ ^ ~~~~~~~~~
<program source>:133:5: warning: comparison of integers of different signs: 'int' and 'const unsigned int'
    def_printFromWord(__global, printFromWord_glbl_n, "\n")
    ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
<program source>:116:27: note: expanded from macro 'def_printFromWord'
        for (int j = 0; j < len_bytes; j++){    \
                        ~ ^ ~~~~~~~~~
<program source>:394:1: warning: comparison of integers of different signs: 'int' and 'const unsigned int'
def_hash(hash_global, __global, __global, md_pad__global, printFromLong_glbl_n)
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
<program source>:330:35: note: expanded from macro 'def_hash'
    for (int block_i = 0; block_i < nBlocks; block_i++)     \
                          ~~~~~~~ ^ ~~~~~~~
<program source>:395:1: warning: comparison of integers of different signs: 'int' and 'const unsigned int'
def_hash(hash_private, __private, __private, md_pad__private, printFromLong_n)
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
<program source>:330:35: note: expanded from macro 'def_hash'
    for (int block_i = 0; block_i < nBlocks; block_i++)     \
                          ~~~~~~~ ^ ~~~~~~~
<program source>:396:1: warning: comparison of integers of different signs: 'int' and 'const unsigned int'
def_hash(hash_glbl_to_priv, __global, __private, md_pad__global, printFromLong_glbl_n)
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
<program source>:330:35: note: expanded from macro 'def_hash'
    for (int block_i = 0; block_i < nBlocks; block_i++)     \
                          ~~~~~~~ ^ ~~~~~~~
<program source>:397:1: warning: comparison of integers of different signs: 'int' and 'const unsigned int'
def_hash(hash_priv_to_glbl, __private, __global, md_pad__private, printFromLong_n)
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
<program source>:330:35: note: expanded from macro 'def_hash'
    for (int block_i = 0; block_i < nBlocks; block_i++)     \
                          ~~~~~~~ ^ ~~~~~~~
<program source>:458:27: warning: comparison of integers of different signs: 'int' and 'unsigned long'
        for (int j = 0; j < end; j++){
                        ~ ^ ~~~
<program source>:488:23: warning: comparison of integers of different signs: 'int' and 'unsigned long'
    for (int j = 0; j < m_len_word; j++){
                    ~ ^ ~~~~~~~~~~
<program source>:630:20: warning: initializing '__global unsigned long *' with an expression of type 'unsigned long const __global[4]' discards qualifiers
    __global word *pwdBuffer = pwdbuffer_arg[0].buffer;
                   ^           ~~~~~~~~~~~~~~~~~~~~~~~

  warn(text, CompilerWarning)

======================================================================
ERROR: test_BIP39_Eth_OpenCL_Brute (__main__.TestRecoveryFromAddress)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/vshuraeff/work/crypto/btcrecover/btcrecover/test/test_seeds.py", line 115, in skip_or_test
    test_func(self)
  File "/Users/vshuraeff/work/crypto/btcrecover/btcrecover/test/test_seeds.py", line 1234, in test_BIP39_Eth_OpenCL_Brute
    self.assertEqual(btcrseed.WalletEthereum._return_verified_password_or_false_opencl(wallet,
  File "/Users/vshuraeff/work/crypto/btcrecover/btcrecover/btcrseed.py", line 1000, in _return_verified_password_or_false_opencl
    clResult = self.opencl_algo.cl_pbkdf2(self.opencl_context_pbkdf2_sha512[i], cleaned_mnemonic_ids_list,
  File "/Users/vshuraeff/work/crypto/btcrecover/lib/opencl_brute/opencl.py", line 705, in cl_pbkdf2
    result = concat(self.opencl_ctx.run(bufStructs, func, iter(passwordlist), salt))
  File "/Users/vshuraeff/work/crypto/btcrecover/lib/opencl_brute/opencl.py", line 524, in concat
    return [obj for lval in ll for obj in lval]
  File "/Users/vshuraeff/work/crypto/btcrecover/lib/opencl_brute/opencl.py", line 524, in <listcomp>
    return [obj for lval in ll for obj in lval]
  File "/Users/vshuraeff/work/crypto/btcrecover/lib/opencl_brute/opencl.py", line 263, in run
    func(self, pwdim, pass_g, salt_g, result_g)
  File "/Users/vshuraeff/work/crypto/btcrecover/lib/opencl_brute/opencl.py", line 702, in func
    prg.pbkdf2(s.queue, pwdim, None, pass_g, salt_g, result_g,
  File "/Users/vshuraeff/.asdf/installs/python/3.9.15/lib/python3.9/site-packages/pyopencl/__init__.py", line 895, in kernel_call
    return self._enqueue(self, queue, global_size, local_size, *args, **kwargs)
  File "<pyopencl invoker for 'pbkdf2'>", line 8, in enqueue_knl_pbkdf2
pyopencl._cl.LogicError: clEnqueueNDRangeKernel failed: INVALID_PROGRAM_EXECUTABLE

Stdout:
Using the 'en' wordlist.
Assuming a 12 word mnemonic. (This can be overridden with --mnemonic-length)
OpenCL: Auto Selecting Best Platform

======================================================================
ERROR: test_Electrum_OpenCL_Brute (__main__.TestRecoveryFromAddress)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/vshuraeff/work/crypto/btcrecover/btcrecover/test/test_seeds.py", line 115, in skip_or_test
    test_func(self)
  File "/Users/vshuraeff/work/crypto/btcrecover/btcrecover/test/test_seeds.py", line 1279, in test_Electrum_OpenCL_Brute
    self.assertEqual(btcrseed.WalletElectrum2._return_verified_password_or_false_opencl(wallet,
  File "/Users/vshuraeff/work/crypto/btcrecover/btcrecover/btcrseed.py", line 1000, in _return_verified_password_or_false_opencl
    clResult = self.opencl_algo.cl_pbkdf2(self.opencl_context_pbkdf2_sha512[i], cleaned_mnemonic_ids_list,
  File "/Users/vshuraeff/work/crypto/btcrecover/lib/opencl_brute/opencl.py", line 705, in cl_pbkdf2
    result = concat(self.opencl_ctx.run(bufStructs, func, iter(passwordlist), salt))
  File "/Users/vshuraeff/work/crypto/btcrecover/lib/opencl_brute/opencl.py", line 524, in concat
    return [obj for lval in ll for obj in lval]
  File "/Users/vshuraeff/work/crypto/btcrecover/lib/opencl_brute/opencl.py", line 524, in <listcomp>
    return [obj for lval in ll for obj in lval]
  File "/Users/vshuraeff/work/crypto/btcrecover/lib/opencl_brute/opencl.py", line 263, in run
    func(self, pwdim, pass_g, salt_g, result_g)
  File "/Users/vshuraeff/work/crypto/btcrecover/lib/opencl_brute/opencl.py", line 702, in func
    prg.pbkdf2(s.queue, pwdim, None, pass_g, salt_g, result_g,
  File "/Users/vshuraeff/.asdf/installs/python/3.9.15/lib/python3.9/site-packages/pyopencl/__init__.py", line 895, in kernel_call
    return self._enqueue(self, queue, global_size, local_size, *args, **kwargs)
  File "<pyopencl invoker for 'pbkdf2'>", line 8, in enqueue_knl_pbkdf2
pyopencl._cl.LogicError: clEnqueueNDRangeKernel failed: INVALID_PROGRAM_EXECUTABLE

Stdout:
Using the 'en' wordlist.
OpenCL: Auto Selecting Best Platform

----------------------------------------------------------------------
Ran 3 tests in 0.235s

FAILED (errors=3)
vshuraeff commented 1 year ago

downgrading pyopencl to 2022.2.2 (pip3 install pyopencl==2022.2.2) gets test_BIP39_BTC_OpenCL_Brute (__main__.TestRecoveryFromAddress) ... [1] 39487 segmentation fault PYOPENCL_COMPILER_OUTPUT=1 python -m btcrecover.test.test_seeds -v

vshuraeff commented 1 year ago

downgrading to pyopencl==2022.1.4 gives the same segmentation fault :( where I need to check, what is going on?

vshuraeff commented 1 year ago

well, egmentation fault gets when shell is not restarted, after restart it gives same INVALID_PROGRAM_EXECUTABLE on almost any versions of pyopencl in 2021 and 2022 series.

very sad :(

vshuraeff commented 1 year ago

downgrading python to 3.8.15 gives the same result on mac with vega

3rdIteration commented 1 year ago

I haven't tested pyopencl functionality on any Mac platform. Do you get the same behavior in Linux?

vshuraeff commented 1 year ago

no, I does not trying to test it under Linux, but as I see, only downgrading python to 3.8 gets working btcrecover to bruteforce, but still cant run any tests

3rdIteration commented 1 year ago

Just use CPU on Mac

vshuraeff commented 1 year ago

I can run it on Mac GPU (Vega 48) with about 900 P/s, but I cant run tests, as I describe above.

image