AttributeError: Can't pickle local object 'construct_web3_formatting_middleware.<locals>.formatter_middleware'

What happened?

I have a problem to parallel executions during querying ethereum node transactions via RPC. Error says that it can not pickle w3 object (when I do pool.starmap_async()). It's important to do in class. If I remove class and do parallel job, making global w3 in calculate_transaction() - it works, but I need object oriented way. If I define w3 outside the class in main(), which way is also good and pass w3 like w.start(w3=w3), error is raised the same. Any ideas how to do?

Code that produced the error

import asyncio
from web3 import Web3, AsyncWeb3
from multiprocessing import Pool, cpu_count

class Wallet():

    def start(self):
        testnet = 'https://eth-pokt.nodies.app'

        # self.w3 = AsyncWeb3(AsyncWeb3.AsyncHTTPProvider(testnet))
        self.w3 = Web3(Web3.HTTPProvider(testnet))
        if not self.w3.is_connected():
            exit("w3 not connected")

        block = self.w3.eth.get_block('latest')
        transactions = block['transactions'][:min(cpu_count(), 10)]

        params = []
        for i, hash_ in enumerate(transactions):
            params.append((i, hash_))

        pool = Pool(min(cpu_count(), 10))
        pool.starmap_async(self.calculate_transaction, params).get()
        pool.close()

    def calculate_transaction(self, i, hash_):
        return self.w3.eth.get_transaction(hash_)

def main():
    w = Wallet()
    w.start()

if __name__ == '__main__':
    main()
    # asyncio.run(main())

Full error output

Traceback (most recent call last):
  File "/home/fridary/python/ethereum/parallel_3.py", line 42, in <module>
    main()
  File "/home/fridary/python/ethereum/parallel_3.py", line 37, in main
    w.start()
  File "/home/fridary/python/ethereum/parallel_3.py", line 26, in start
    pool.starmap_async(self.calculate_transaction, params).get()
  File "/home/fridary/miniconda3/envs/eth/lib/python3.11/multiprocessing/pool.py", line 774, in get
    raise self._value
  File "/home/fridary/miniconda3/envs/eth/lib/python3.11/multiprocessing/pool.py", line 540, in _handle_tasks
    put(task)
  File "/home/fridary/miniconda3/envs/eth/lib/python3.11/multiprocessing/connection.py", line 206, in send
    self._send_bytes(_ForkingPickler.dumps(obj))
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/fridary/miniconda3/envs/eth/lib/python3.11/multiprocessing/reduction.py", line 51, in dumps
    cls(buf, protocol).dump(obj)
AttributeError: Can't pickle local object 'construct_web3_formatting_middleware.<locals>.formatter_middleware'

Fill this section in if you know how this could or should be fixed

No response

web3 Version

6.16.0

Python Version

3.11.8

Operating System

Ubuntu 20.04.6 LTS

Output from `pip freeze`

aiohttp==3.9.3
aiosignal==1.3.1
alchemy-sdk-py==0.2.0
attributedict==0.3.0
attrs==23.2.0
bitarray==2.9.2
blessings==1.7
cachetools==5.3.3
certifi==2024.2.2
chardet==5.2.0
charset-normalizer==3.3.2
codecov==2.1.13
colorama==0.4.6
coloredlogs==15.0.1
colour-runner==0.1.1
coverage==7.4.4
cytoolz==0.12.3
deepdiff==6.7.1
distlib==0.3.8
eth-abi==4.2.1
eth-account==0.11.0
eth-hash==0.7.0
eth-keyfile==0.8.0
eth-keys==0.5.0
eth-rlp==1.0.1
eth-typing==4.0.0
eth-utils==4.0.0
filelock==3.13.3
frozendict==2.3.10
frozenlist==1.4.1
hexbytes==0.3.1
humanfriendly==10.0
idna==3.6
inspecta==0.1.3
jsonschema==4.21.1
jsonschema-specifications==2023.12.1
lru-dict==1.2.0
markdown-it-py==3.0.0
mdurl==0.1.2
moralis==0.1.45
multidict==6.0.5
numpy==1.26.4
ordered-set==4.1.0
packaging==24.0
pandas==2.2.1
parsimonious==0.9.0
platformdirs==4.2.0
pluggy==1.4.0
protobuf==5.26.1
pycryptodome==3.20.0
Pygments==2.17.2
pyproject-api==1.6.1
python-dateutil==2.8.2
python-dotenv==1.0.1
pytz==2024.1
pyunormalize==15.1.0
referencing==0.34.0
regex==2023.12.25
requests==2.31.0
rich==13.7.1
rlp==4.0.0
rootpath==0.1.1
rpds-py==0.18.0
six==1.16.0
tabulate==0.9.0
termcolor==2.4.0
toolz==0.12.1
tox==4.14.2
typing_extensions==4.3.0
tzdata==2024.1
urllib3==1.26.18
virtualenv==20.25.1
web3==6.16.0
web3-input-decoder==0.1.11
websockets==12.0
yarl==1.9.4

ethereum / web3.py