Can't pickle/dill contract objects " 'web3._utils.datatypes' has no attribute 'Contract'"

[guillaumecote]

• Version: 5.6.0
• Python: 3.6.2
• OS: win
• pip freeze output:

0x-contract-addresses==3.0.0
0x-contract-artifacts==3.0.0
0x-contract-wrappers==2.0.0
0x-json-schemas==2.1.0
0x-order-utils==4.0.0
address==0.1.1
aiohttp==3.5.4
alabaster==0.7.10
asn1crypto==0.23.0
astroid==1.6.1
async-timeout==3.0.1
attrdict==2.0.1
attrs==19.3.0
autobahn==17.10.1
Automat==0.6.0
Babel==2.5.3
base58==2.0.0
beautifulsoup4==4.6.0
bitcoin==1.1.42
bleach==2.1.2
certifi==2019.11.28
cffi==1.11.2
chardet==3.0.4
checksumdir==1.1.7
cloudpickle==0.5.2
coincurve==12.0.0
colorama==0.3.9
constantly==15.1.0
cryptography==2.1.4
cycler==0.10.0
cytoolz==0.10.1
dateparser==0.6.0
decorator==4.1.2
Deprecated==1.2.7
dill==0.3.1.1
Django==2.0.2
docutils==0.14
EasyProcess==0.2.7
ecdsa==0.15
entrypoints==0.2.3
eth-abi==2.1.1
eth-account==0.4.0
eth-hash==0.2.0
eth-keyfile==0.5.1
eth-keys==0.3.1
eth-rlp==0.1.2
eth-typing==2.2.1
eth-utils==1.8.4
ethereum==2.1.2
ethereum-abi-utils==0.4.4
ethereum-keyfile==0.3.0
ethereum-keys==0.1.0a7
ethereum-tester==0.1.0b2
ethereum-utils==0.5.1
ftfy==5.3.0
future==0.17.1
gevent==1.2.2
greenlet==0.4.12
hexbytes==0.2.0
html5lib==1.0.1
hyperlink==17.3.1
idna==2.9
idna-ssl==1.1.0
imagesize==0.7.1
importlib-metadata==1.5.0
incremental==17.5.0
ipfshttpclient==0.4.12
ipykernel==4.8.0
ipython==6.1.0
ipython-genutils==0.2.0
isort==4.3.1
jedi==0.10.2
Jinja2==2.10
joblib==0.13.2
JsonForm==0.0.2
jsonrpc==1.2
jsonschema==3.2.0
JsonSir==0.0.2
jupyter-client==5.2.2
jupyter-core==4.4.0
langcodes==1.4.1
lazy-object-proxy==1.3.1
lru-dict==1.1.6
marisa-trie==0.7.4
MarkupSafe==1.0
matplotlib==2.1.2
mccabe==0.6.1
mistune==0.8.3
MouseInfo==0.0.4
mpmath==1.1.0
msgpack==0.5.6
multiaddr==0.0.9
multidict==4.5.2
mypy-extensions==0.4.3
nbconvert==5.3.1
nbformat==4.4.0
netaddr==0.7.19
numpy==1.14.0
numpydoc==0.7.0
oauthlib==2.0.6
opencv-python==4.1.0.25
pandocfilters==1.4.2
parsimonious==0.8.1
pbkdf2==1.3
pickleshare==0.7.4
pika==1.1.0
Pillow==6.1.0
praw==5.3.0
prawcore==0.13.0
prompt-toolkit==1.0.15
protobuf==3.11.3
psutil==5.4.3
PTable==0.9.2
py-ecc==1.1.3
py-geth==1.10.1
pyap==0.2.0
pyasn1==0.4.2
pyasn1-modules==0.2.1
PyAutoGUI==0.9.47
pycodestyle==2.3.1
pycparser==2.18
pycryptodome==3.9.7
pyethereum==1.0.0
pyflakes==1.6.0
PyGetWindow==0.0.7
Pygments==2.2.0
pylint==1.8.2
pylru==1.0.9
pymongo==3.6.0
PyMsgBox==1.0.7
PyMySQL==0.8.0
pyOpenSSL==17.5.0
pyparsing==2.2.0
pyperclip==1.7.0
pypiwin32==223
PyQt5==5.10
PyRect==0.1.4
pyrsistent==0.15.7
pyscreenshot==0.5.1
PyScreeze==0.1.22
pysha3==1.0.2
python-binance==0.6.3
python-dateutil==2.6.1
Python-EasyConfig==0.1.7
python3-anticaptcha==1.4
PyTweening==1.0.3
pytz==2017.3
pywin32==227
pyyaml==3.12
pyzmq==16.0.4
QtAwesome==0.4.4
qtconsole==4.3.1
QtPy==1.3.1
regex==2017.7.28
repoze.lru==0.7
requests==2.23.0
requests-oauthlib==0.8.0
Resource==0.2.1
rlp==1.2.0
rope==0.10.7
ruamel.yaml==0.15.35
scipy==1.4.1
scrypt==0.8.13
selenium==3.141.0
semantic-version==2.6.0
service-identity==17.0.0
sigfig==1.1.8
simplegeneric==0.8.1
simplejson==3.11.1
sip==4.19.7
six==1.14.0
snowballstemmer==1.2.1
socketIO-client==0.7.2
socketIO-client-nexus==0.7.6
sortedcontainers==2.1.0
Sphinx==1.6.7
sphinxcontrib-websupport==1.0.1
spyder==3.2.6
stringcase==1.2.0
style==1.1.0
sympy==1.4
testpath==0.3.1
toolz==0.10.0
tornado==4.5.3
traitlets==4.3.2
tweepy==3.5.0
Twisted==17.9.0
txaio==2.8.2
typing-extensions==3.7.4.1
tzlocal==1.5.1
Unidecode==1.1.1
update==0.0.1
update-checker==0.16
urllib3==1.25.8
varint==1.0.2
wcwidth==0.1.7
web3==5.6.0
webencodings==0.5.1
websocket==0.2.1
websocket-client==0.44.0
websockets==8.1
wordfreq==1.7.0
wrapt==1.10.11
yarl==1.3.0
zipp==3.1.0
zope.interface==4.4.3

What was wrong?

I'm trying to find any way to save web3.contracts to limit the amount of calls I generate through infura during code initialization phase.

• The code which produced the error

  
  import json
  infura = "https://mainnet.infura.io/v3/"+id

w3 = Web3(HTTPProvider(infura)) abi = {} with open('uniswap_factory.json') as f: abi['factory'] = json.load(f) contract = w3.eth.contract(address="0xc0a47dFe034B400B47bDaD5FecDa2621de6c4d95", abi = abi['factory']) dill.dumps(contract)

The same seems to happen with any contract.

* The full output of the error

---

AttributeError Traceback (most recent call last) c:\users\sir_s\appdata\local\programs\python\python36-32\lib\pickle.py in _getattribute(obj, name) 268 parent = obj --> 269 obj = getattr(obj, subpath) 270 except AttributeError:

AttributeError: module 'web3._utils.datatypes' has no attribute 'Contract'

During handling of the above exception, another exception occurred:

AttributeError Traceback (most recent call last) c:\users\sir_s\appdata\local\programs\python\python36-32\lib\pickle.py in save_global(self, obj, name) 917 module = sys.modules[module_name] --> 918 obj2, parent = _getattribute(module, name) 919 except (ImportError, KeyError, AttributeError):

c:\users\sir_s\appdata\local\programs\python\python36-32\lib\pickle.py in _getattribute(obj, name) 271 raise AttributeError("Can't get attribute {!r} on {!r}" --> 272 .format(name, obj)) 273 return obj, parent

AttributeError: Can't get attribute 'Contract' on <module 'web3._utils.datatypes' from 'C:\Users\Sir_s\AppData\Roaming\Python\Python36\site-packages\web3\_utils\datatypes.py'>

During handling of the above exception, another exception occurred:

PicklingError Traceback (most recent call last)

in () 8 abi['factory'] = json.load(f) 9 contract = w3.eth.contract(address="0xc0a47dFe034B400B47bDaD5FecDa2621de6c4d95", abi = abi['factory']) ---> 10 dill.dumps(contract) c:\users\sir_s\appdata\local\programs\python\python36-32\lib\site-packages\dill\_dill.py in dumps(obj, protocol, byref, fmode, recurse, **kwds) 263 """pickle an object to a string""" 264 file = StringIO() --> 265 dump(obj, file, protocol, byref, fmode, recurse, **kwds)#, strictio) 266 return file.getvalue() 267 c:\users\sir_s\appdata\local\programs\python\python36-32\lib\site-packages\dill\_dill.py in dump(obj, file, protocol, byref, fmode, recurse, **kwds) 257 _kwds = kwds.copy() 258 _kwds.update(dict(byref=byref, fmode=fmode, recurse=recurse)) --> 259 Pickler(file, protocol, **_kwds).dump(obj) 260 return 261 c:\users\sir_s\appdata\local\programs\python\python36-32\lib\site-packages\dill\_dill.py in dump(self, obj) 443 raise PicklingError(msg) 444 else: --> 445 StockPickler.dump(self, obj) 446 stack.clear() # clear record of 'recursion-sensitive' pickled objects 447 return c:\users\sir_s\appdata\local\programs\python\python36-32\lib\pickle.py in dump(self, obj) 407 if self.proto >= 4: 408 self.framer.start_framing() --> 409 self.save(obj) 410 self.write(STOP) 411 self.framer.end_framing() c:\users\sir_s\appdata\local\programs\python\python36-32\lib\pickle.py in save(self, obj, save_persistent_id) 519 520 # Save the reduce() output and finally memoize the object --> 521 self.save_reduce(obj=obj, *rv) 522 523 def persistent_id(self, obj): c:\users\sir_s\appdata\local\programs\python\python36-32\lib\pickle.py in save_reduce(self, func, args, state, listitems, dictitems, obj) 603 "args[0] from __newobj__ args has the wrong class") 604 args = args[1:] --> 605 save(cls) 606 save(args) 607 write(NEWOBJ) c:\users\sir_s\appdata\local\programs\python\python36-32\lib\pickle.py in save(self, obj, save_persistent_id) 474 f = self.dispatch.get(t) 475 if f is not None: --> 476 f(self, obj) # Call unbound method with explicit self 477 return 478 c:\users\sir_s\appdata\local\programs\python\python36-32\lib\site-packages\dill\_dill.py in save_type(pickler, obj) 1354 #print ("%s\n%s" % (type(obj), obj.__name__)) 1355 #print ("%s\n%s" % (obj.__bases__, obj.__dict__)) -> 1356 StockPickler.save_global(pickler, obj) 1357 log.info("# T4") 1358 return c:\users\sir_s\appdata\local\programs\python\python36-32\lib\pickle.py in save_global(self, obj, name) 920 raise PicklingError( 921 "Can't pickle %r: it's not found as %s.%s" % --> 922 (obj, module_name, name)) 923 else: 924 if obj2 is not obj: PicklingError: Can't pickle : it's not found as web3._utils.datatypes.Contract ``` * What type of node you were connecting to. Infura HTTP ### How can it be fixed? I'm a bit lost, Contract isn't explicitly defined in web3._utils.datatypes...

[pipermerriam]

The Contract classes (and instances) are going to be unpickleable since they are dynamically created classes.

In order to support pickle-ability the Web3 class would need to be pickleable (which I currently would not be surprised if it wasnt). After that we'd need to introduce the pickle APIs for handling custom construction/deconstruction of objects.

This should be a relatively straight forward thing to do for anyone with a little python experience.

[lforet]

Has anyone made progress on this.. I would PAY for this functionality - to be able to store web3 smart contacts in a local mysql db

[kclowes]

This hasn't reached the top of our priority queue, so no progress has been made. If you'd like to take a crack at it, @lforet, we'd be happy to review! That's probably the fastest way to get it in at this point.

[journeytosilius]

I stand corrected. What I have done is to abstract one level above the class that contains the method that calls the Contract object and used Pathos multiprocessing instead of the in-built python multiprocessing. It works now.

[fridary]

I need to store w3 contracts in files also for speed up backtesting, please add support to this