search

lbryio / lbry-sdk

The LBRY SDK for building decentralized, censorship resistant, monetized, digital content apps.
https://lbry.com
MIT License
7.2k stars 483 forks source link

Various tests failing when run on Ubuntu 22.04.1: "ValueError: unsupported hash type ripemd160" #3710

Closed moodyjon closed 1 year ago

moodyjon commented 1 year ago

I just started seeing this today in some of the CI runs on my pull requests.

https://github.com/lbryio/lbry-sdk/actions/runs/3515232576/jobs/5890229952

Current theory is some of the Github runners are Ubuntu 22.04.1 now, and the OpenSSL library is not providing ripemd160 anymore in Ubuntu 22.04.1.

image

https://github.com/openssl/openssl/issues/16994

I have seen passing runs on Ubuntu 20.04.5.

Example failure:

======================================================================
[437](https://github.com/lbryio/lbry-sdk/actions/runs/3515232576/jobs/5890229952#step:9:438)
ERROR: test_download_data_timeout (tests.unit.stream.test_stream_manager.TestStreamManager)
[438](https://github.com/lbryio/lbry-sdk/actions/runs/3515232576/jobs/5890229952#step:9:439)
----------------------------------------------------------------------
[439](https://github.com/lbryio/lbry-sdk/actions/runs/3515232576/jobs/5890229952#step:9:440)
Traceback (most recent call last):
[440](https://github.com/lbryio/lbry-sdk/actions/runs/3515232576/jobs/5890229952#step:9:441)
  File "/opt/hostedtoolcache/Python/3.7.15/x64/lib/python3.7/hashlib.py", line 150, in __hash_new
[441](https://github.com/lbryio/lbry-sdk/actions/runs/3515232576/jobs/5890229952#step:9:442)
    return _hashlib.new(name, data)
[442](https://github.com/lbryio/lbry-sdk/actions/runs/3515232576/jobs/5890229952#step:9:443)
ValueError: [digital envelope routines] initialization error
[443](https://github.com/lbryio/lbry-sdk/actions/runs/3515232576/jobs/5890229952#step:9:444)

[444](https://github.com/lbryio/lbry-sdk/actions/runs/3515232576/jobs/5890229952#step:9:445)
During handling of the above exception, another exception occurred:
[445](https://github.com/lbryio/lbry-sdk/actions/runs/3515232576/jobs/5890229952#step:9:446)

[446](https://github.com/lbryio/lbry-sdk/actions/runs/3515232576/jobs/5890229952#step:9:447)
Traceback (most recent call last):
[447](https://github.com/lbryio/lbry-sdk/actions/runs/3515232576/jobs/5890229952#step:9:448)
  File "/home/runner/work/lbry-sdk/lbry-sdk/lbry/testcase.py", line 145, in run
[448](https://github.com/lbryio/lbry-sdk/actions/runs/3515232576/jobs/5890229952#step:9:449)
    self.loop.run_until_complete(maybe_coroutine)
[449](https://github.com/lbryio/lbry-sdk/actions/runs/3515232576/jobs/5890229952#step:9:450)
  File "/opt/hostedtoolcache/Python/3.7.15/x64/lib/python3.7/asyncio/base_events.py", line 587, in run_until_complete
[450](https://github.com/lbryio/lbry-sdk/actions/runs/3515232576/jobs/5890229952#step:9:451)
    return future.result()
[451](https://github.com/lbryio/lbry-sdk/actions/runs/3515232576/jobs/5890229952#step:9:452)
  File "/home/runner/work/lbry-sdk/lbry-sdk/tests/unit/stream/test_stream_manager.py", line 394, in test_download_data_timeout
[452](https://github.com/lbryio/lbry-sdk/actions/runs/3515232576/jobs/5890229952#step:9:453)
    await self.setup_stream_manager()
[453](https://github.com/lbryio/lbry-sdk/actions/runs/3515232576/jobs/5890229952#step:9:454)
  File "/home/runner/work/lbry-sdk/lbry-sdk/tests/unit/stream/test_stream_manager.py", line 143, in setup_stream_manager
[454](https://github.com/lbryio/lbry-sdk/actions/runs/3515232576/jobs/5890229952#step:9:455)
    balance, fee)
[455](https://github.com/lbryio/lbry-sdk/actions/runs/3515232576/jobs/5890229952#step:9:456)
  File "/home/runner/work/lbry-sdk/lbry-sdk/tests/unit/stream/test_stream_manager.py", line 105, in get_mock_wallet
[456](https://github.com/lbryio/lbry-sdk/actions/runs/3515232576/jobs/5890229952#step:9:457)
    wallet.generate_account(ledger)
[457](https://github.com/lbryio/lbry-sdk/actions/runs/3515232576/jobs/5890229952#step:9:458)
  File "/home/runner/work/lbry-sdk/lbry-sdk/lbry/wallet/wallet.py", line 87, in generate_account
[458](https://github.com/lbryio/lbry-sdk/actions/runs/3515232576/jobs/5890229952#step:9:459)
    return Account.generate(ledger, self)
[459](https://github.com/lbryio/lbry-sdk/actions/runs/3515232576/jobs/5890229952#step:9:460)
  File "/home/runner/work/lbry-sdk/lbry-sdk/lbry/wallet/account.py", line 310, in generate
[460](https://github.com/lbryio/lbry-sdk/actions/runs/3515232576/jobs/5890229952#step:9:461)
    'address_generator': address_generator or {}
[461](https://github.com/lbryio/lbry-sdk/actions/runs/3515232576/jobs/5890229952#step:9:462)
  File "/home/runner/work/lbry-sdk/lbry-sdk/lbry/wallet/account.py", line 343, in from_dict
[462](https://github.com/lbryio/lbry-sdk/actions/runs/3515232576/jobs/5890229952#step:9:463)
    name = f'Account #{public_key.address}'
[463](https://github.com/lbryio/lbry-sdk/actions/runs/3515232576/jobs/5890229952#step:9:464)
  File "/home/runner/work/lbry-sdk/lbry-sdk/lbry/wallet/util.py", line 54, in __get__
[464](https://github.com/lbryio/lbry-sdk/actions/runs/3515232576/jobs/5890229952#step:9:465)
    value = self.f(obj)
[465](https://github.com/lbryio/lbry-sdk/actions/runs/3515232576/jobs/5890229952#step:9:466)
  File "/home/runner/work/lbry-sdk/lbry-sdk/lbry/wallet/bip32.py", line 118, in address
[466](https://github.com/lbryio/lbry-sdk/actions/runs/3515232576/jobs/5890229952#step:9:467)
    return self.ledger.public_key_to_address(self.pubkey_bytes)
[467](https://github.com/lbryio/lbry-sdk/actions/runs/3515232576/jobs/5890229952#step:9:468)
  File "/home/runner/work/lbry-sdk/lbry-sdk/lbry/wallet/ledger.py", line 202, in public_key_to_address
[468](https://github.com/lbryio/lbry-sdk/actions/runs/3515232576/jobs/5890229952#step:9:469)
    return cls.hash160_to_address(hash160(public_key))
[469](https://github.com/lbryio/lbry-sdk/actions/runs/3515232576/jobs/5890229952#step:9:470)
  File "/home/runner/work/lbry-sdk/lbry-sdk/lbry/crypto/hash.py", line 36, in hash160
[470](https://github.com/lbryio/lbry-sdk/actions/runs/3515232576/jobs/5890229952#step:9:471)
    return ripemd160(sha256(x))
[471](https://github.com/lbryio/lbry-sdk/actions/runs/3515232576/jobs/5890229952#step:9:472)
  File "/home/runner/work/lbry-sdk/lbry-sdk/lbry/crypto/hash.py", line 18, in ripemd160
[472](https://github.com/lbryio/lbry-sdk/actions/runs/3515232576/jobs/5890229952#step:9:473)
    h = hashlib.new('ripemd160')
[473](https://github.com/lbryio/lbry-sdk/actions/runs/3515232576/jobs/5890229952#step:9:474)
  File "/opt/hostedtoolcache/Python/3.7.15/x64/lib/python3.7/hashlib.py", line 156, in __hash_new
[474](https://github.com/lbryio/lbry-sdk/actions/runs/3515232576/jobs/5890229952#step:9:475)
    return __get_builtin_constructor(name)(data)
[475](https://github.com/lbryio/lbry-sdk/actions/runs/3515232576/jobs/5890229952#step:9:476)
  File "/opt/hostedtoolcache/Python/3.7.15/x64/lib/python3.7/hashlib.py", line 113, in __get_builtin_constructor
[476](https://github.com/lbryio/lbry-sdk/actions/runs/3515232576/jobs/5890229952#step:9:477)
    raise ValueError('unsupported hash type ' + name)
[477](https://github.com/lbryio/lbry-sdk/actions/runs/3515232576/jobs/5890229952#step:9:478)
ValueError: unsupported hash type ripemd160
[478](https://github.com/lbryio/lbry-sdk/actions/runs/3515232576/jobs/5890229952#step:9:479)
moodyjon commented 1 year ago

https://github.com/openssl/openssl/issues/16994#issuecomment-1257277404

I haven't figured out what the consensus is on the best solution. People are talking about adding their own native python implementation.

moodyjon commented 1 year ago

OpenSSL 3.0.7 may be reversing the deprecation. https://www.openssl.org/blog/blog/2022/10/18/rmd160-and-the-legacy-provider/

See also: openssl/openssl pull 19375

moodyjon commented 1 year ago

Alternative pure python implementation: https://github.com/symbol/ripemd https://pypi.org/project/ripemd-hash/

drzraf commented 7 months ago

See https://github.com/python/cpython/issues/92876 about another possibility to fix this