ethereum / web3.py

A python interface for interacting with the Ethereum blockchain and ecosystem.
http://web3py.readthedocs.io
MIT License
4.95k stars 1.69k forks source link

Async websocket subscription using 100% CPU #3305

Closed jonasbrami closed 5 months ago

jonasbrami commented 5 months ago
(web3) quenouille@LAPTOP-G5F7OIM4:~/web3.py$ pip freeze
aiohttp==3.9.3
aiosignal==1.3.1
alabaster==0.7.16
annotated-types==0.6.0
attrs==23.2.0
Babel==2.14.0
bitarray==2.9.2
black==24.3.0
blocklint==0.2.4
build==1.1.1
bump2version==1.0.1
bumpversion==0.6.0
cached-property==1.5.2
cachetools==5.3.3
certifi==2024.2.2
cffi==1.16.0
chardet==5.2.0
charset-normalizer==3.3.2
click==8.1.7
click-default-group==1.2.4
colorama==0.4.6
cryptography==42.0.5
cytoolz==0.12.3
distlib==0.3.8
docopt==0.6.2
docutils==0.20.1
eth-account==0.11.0
eth-bloom==3.0.0
eth-hash==0.7.0
eth-keyfile==0.8.0
eth-keys==0.5.0
eth-rlp==1.0.1
eth-tester==0.10.0b3
eth-typing==4.0.0
eth-utils==4.0.0
eth_abi==5.0.1
execnet==2.0.2
filelock==3.13.3
flake8==3.8.3
flaky==3.8.1
frozenlist==1.4.1
hexbytes==0.3.1
hypothesis==6.99.13
idna==3.6
imagesize==1.4.1
importlib_metadata==7.1.0
incremental==22.10.0
iniconfig==2.0.0
isort==5.13.2
jaraco.classes==3.3.1
jaraco.context==4.3.0
jaraco.functools==4.0.0
jeepney==0.8.0
Jinja2==3.1.3
keyring==25.0.0
lru-dict==1.3.0
markdown-it-py==3.0.0
MarkupSafe==2.1.5
mccabe==0.6.1
mdurl==0.1.2
more-itertools==10.2.0
multidict==6.0.5
mypy==1.4.1
mypy-extensions==1.0.0
nh3==0.2.17
packaging==24.0
parsimonious==0.9.0
pathspec==0.12.1
pkginfo==1.10.0
platformdirs==4.2.0
pluggy==1.4.0
py-ecc==7.0.0
py-evm==0.9.0b1
py-geth==4.3.0
pycodestyle==2.6.0
pycparser==2.21
pycryptodome==3.20.0
pydantic==2.6.4
pydantic_core==2.16.3
pyflakes==2.2.0
Pygments==2.17.2
pyproject-api==1.6.1
pyproject_hooks==1.0.0
pytest==8.1.1
pytest-asyncio==0.21.1
pytest-mock==3.14.0
pytest-watch==4.2.0
pytest-xdist==3.5.0
pyunormalize==15.1.0
readme_renderer==43.0
regex==2023.12.25
requests==2.31.0
requests-toolbelt==1.0.0
rfc3986==2.0.0
rich==13.7.1
rlp==4.0.0
safe-pysha3==1.0.4
SecretStorage==3.3.3
semantic-version==2.10.0
snowballstemmer==2.2.0
sortedcontainers==2.4.0
Sphinx==7.2.6
sphinx-rtd-theme==2.0.0
sphinxcontrib-applehelp==1.0.8
sphinxcontrib-devhelp==1.0.6
sphinxcontrib-htmlhelp==2.0.5
sphinxcontrib-jquery==4.1
sphinxcontrib-jsmath==1.0.1
sphinxcontrib-qthelp==1.0.7
sphinxcontrib-serializinghtml==1.1.10
tomli==2.0.1
toolz==0.12.1
towncrier==21.9.0
tox==4.14.2
tqdm==4.66.2
trie==3.0.0
twine==5.0.0
types-requests==2.31.0.20240311
types-setuptools==69.2.0.20240317
typing_extensions==4.10.0
urllib3==2.2.1
virtualenv==20.25.1
watchdog==4.0.0
-e git+ssh://git@github.com/ethereum/web3.py.git@653c536d5a2fbb31b231d39948bac1244ee622c3#egg=web3
websockets==12.0
yarl==1.9.4
zipp==3.18.1

What was wrong?

Please include any of the following that are applicable:

WS_PROVIDER_URL = "wss://eth-mainnet.ws.alchemyapi.io/ws/demo"

async def subscribe_to_new_block_head(): async with AsyncWeb3(WebSocketProvider(WS_PROVIDER_URL)) as w3: await w3.eth.subscribe("newHeads") async for response in w3.socket.process_subscriptions(): print(response)

if name == "main": asyncio.run(subscribe_to_new_block_head())



* The full output of the error
Not an error, but 100% CPU usage can be observed

* What type of node you were connecting to.
Ethereum mainnet

### How can it be fixed?

---
Making RequestProcessor.pop_raw_response an async function, await pop_raw_response instead of polling it.

**Note:** We prefer to use issues to track our work. If you think you've encountered a bug in web3py or 
have a feature request, you're in the right place. If you have implementation or usage questions, 
please refer to our [documentation](https://web3py.readthedocs.io/en/latest/) and/or join the conversation 
on [discord](https://discord.gg/GHryRvPB84).
jonasbrami commented 5 months ago

Opened a pull request to fix https://github.com/ethereum/web3.py/pull/3306