web3.eth.get_block('latest') fails with Polygon Amoy

[jdbertron]

What happened?

I switched to the Polygon Amoy Testnet . I also tested using the latest Web package 6.17.2 .

Code that produced the error

block: Web3 = await web3.eth.get_block('latest')

Full error output

File "/home/jdbertron/.virtualenvs/bqethfarmer/lib/python3.8/site-packages/eth_utils/applicators.py", line 84, in apply_formatters_to_dict
    yield key, formatters[key](item)
  File "cytoolz/functoolz.pyx", line 263, in cytoolz.functoolz.curry.__call__
  File "/home/jdbertron/.virtualenvs/bqethfarmer/lib/python3.8/site-packages/eth_utils/applicators.py", line 72, in apply_formatter_if
    return formatter(value)
  File "cytoolz/functoolz.pyx", line 263, in cytoolz.functoolz.curry.__call__
  File "/home/jdbertron/.virtualenvs/bqethfarmer/lib/python3.8/site-packages/web3/_utils/method_formatters.py", line 147, in to_hexbytes
    raise ValueError(
ValueError: The value HexBytes('0xd78301030083626f7288676f312e32322e32856c696e75780000000000000000c480c2c0c011798d1fae010039489b1a2b696974981346567abdb98ae389332d22886cb54d312877a919b59c2160a946e293cb985482a5c5621d6e70ec4275891f958204a900') is 102 bytes, but should be 32

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/jdbertron/repositories/bqeth-solver/farmer-listener.py", line 1065, in main
    loop.run_until_complete(
  File "/usr/lib64/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/home/jdbertron/repositories/bqeth-solver/farmer-listener.py", line 867, in event_loop
    block: Web3 = await web3.eth.get_block('latest')
  File "/home/jdbertron/.virtualenvs/bqethfarmer/lib/python3.8/site-packages/web3/eth/async_eth.py", line 442, in get_block
    return await self._get_block(block_identifier, full_transactions)
  File "/home/jdbertron/.virtualenvs/bqethfarmer/lib/python3.8/site-packages/web3/module.py", line 121, in caller
    return apply_result_formatters(result_formatters, result)
  File "cytoolz/functoolz.pyx", line 263, in cytoolz.functoolz.curry.__call__
  File "/home/jdbertron/.virtualenvs/bqethfarmer/lib/python3.8/site-packages/web3/module.py", line 49, in apply_result_formatters
    formatted_result = pipe(result, result_formatters)
  File "cytoolz/functoolz.pyx", line 680, in cytoolz.functoolz.pipe
  File "cytoolz/functoolz.pyx", line 655, in cytoolz.functoolz.c_pipe
  File "cytoolz/functoolz.pyx", line 263, in cytoolz.functoolz.curry.__call__
  File "/home/jdbertron/.virtualenvs/bqethfarmer/lib/python3.8/site-packages/eth_utils/applicators.py", line 72, in apply_formatter_if
    return formatter(value)
  File "cytoolz/functoolz.pyx", line 263, in cytoolz.functoolz.curry.__call__
  File "/home/jdbertron/.virtualenvs/bqethfarmer/lib/python3.8/site-packages/web3/_utils/method_formatters.py", line 164, in type_aware_apply_formatters_to_dict
    formatted_dict: Dict[str, Any] = apply_formatters_to_dict(formatters, dict(value))
  File "cytoolz/functoolz.pyx", line 263, in cytoolz.functoolz.curry.__call__
  File "/home/jdbertron/.virtualenvs/bqethfarmer/lib/python3.8/site-packages/eth_utils/functional.py", line 45, in inner
    return callback(fn(*args, **kwargs))
  File "/home/jdbertron/.virtualenvs/bqethfarmer/lib/python3.8/site-packages/eth_utils/applicators.py", line 90, in apply_formatters_to_dict
    raise ValueError(new_error_message) from exc
ValueError: Could not format invalid value '0xd78301030083626f7288676f312e32322e32856c696e75780000000000000000c480c2c0c011798d1fae010039489b1a2b696974981346567abdb98ae389332d22886cb54d312877a919b59c2160a946e293cb985482a5c5621d6e70ec4275891f958204a900' as field 'extraData'
python-BaseException

Fill this section in if you know how this could or should be fixed

I think the extraData 'field' seems to indicate a problem with handling the newly added EIP-4844 blob support.

web3 Version

6.17.2

Python Version

3.8.17

Operating System

Centos 8 Linux

Output from pip freeze

aiohttp==3.8.6
aiosignal==1.3.1
anytree==2.12.0
async-timeout==4.0.3
attrs==23.1.0
bitarray==2.8.2
certifi==2023.7.22
charset-normalizer==3.3.2
cytoolz==0.12.2
eth-abi==4.2.1
eth-account==0.10.0
eth-hash==0.5.2
eth-keyfile==0.6.1
eth-keys==0.4.0
eth-rlp==0.3.0
eth-typing==3.5.2
eth-utils==2.3.1
evdev==1.6.1
frozenlist==1.4.0
future-fstrings==1.2.0
hexbytes==0.3.1
idna==3.4
importlib-resources==6.1.1
jsonschema==4.19.2
jsonschema-specifications==2023.7.1
lru-dict==1.2.0
merklelib==1.0
multidict==6.0.4
parsimonious==0.9.0
pkgutil_resolve_name==1.3.10
protobuf==4.25.0
pycryptodome==3.19.0
pynput==1.7.6
python-dotenv==1.0.0
python-xlib==0.33
pyunormalize==15.0.0
redis==5.0.1
referencing==0.30.2
regex==2023.10.3
requests==2.31.0
rlp==3.0.0
rpds-py==0.12.0
six==1.16.0
toolz==0.12.0
typing_extensions==4.8.0
ujson==5.4.0
urllib3==2.0.7
web3==6.17.2
websockets==12.0
yarl==1.9.2
zipp==3.17.0

[fselmo]

@jdbertron extraData validation can be overridden by injecting the appropriate middleware. Please refer to the docs here for how to achieve this using web3.py v6: https://web3py.readthedocs.io/en/stable/middleware.html#proof-of-authority