pickle.PicklingError when running in cluster & locally

[jadechip]

Hi there! I've deployed the service in a cluster and I get the following error in response to certain API calls such as:

curl --location --request POST 'http://localhost:8000' \
  --header 'Content-Type: application/json' \
  --data-raw '{
      "method": "runtime_getState",
      "id": 1,
      "jsonrpc": "2.0",
      "params": ["System", "Account", ["xxxxxxxx"]]

I've verified that this also happens locally, using docker-compose. I've posted the error below. Please advise.

Thank you 🙏

Traceback (most recent call last):
  File "/usr/local/lib/python3.7/site-packages/gunicorn/workers/sync.py", line 135, in handle
    self.handle_request(listener, req, client, addr)
  File "/usr/local/lib/python3.7/site-packages/gunicorn/workers/sync.py", line 176, in handle_request
    respiter = self.wsgi(environ, resp.start_response)
  File "/usr/local/lib/python3.7/site-packages/falcon/api.py", line 269, in __call__
    responder(req, resp, **params)
  File "/home/substrate-api/app/app/resources/jsonrpc.py", line 182, in on_post
    block_hash=self.block_hash
  File "/home/substrate-api/app/py-substrate-interface/substrateinterface/__init__.py", line 1105, in get_runtime_state
    self.init_runtime(block_hash=block_hash)
  File "/home/substrate-api/app/py-substrate-interface/substrateinterface/__init__.py", line 1034, in init_runtime
    self.cache_region.set('METADATA_{}'.format(self.runtime_version), self.metadata_decoder)
  File "/usr/local/lib/python3.7/site-packages/dogpile/cache/region.py", line 1030, in set
    self.backend.set(key, self._value(value))
  File "/usr/local/lib/python3.7/site-packages/dogpile/cache/backends/redis.py", line 163, in set
    self.client.set(key, pickle.dumps(value, pickle.HIGHEST_PROTOCOL))
_pickle.PicklingError: Can't pickle <class 'abc.(BlockNumber, u32)'>: attribute lookup (BlockNumber, u32) on abc failed

[arjanz]

I temporary disabled the caching of the metadata object in Redis with Dogpile, this led to the pickling errors you encountered. also I updated the application to the latest version of scalecodec and substrate-interface in: https://github.com/polkascan/substrate-interface-api/blob/af672d190ca7e6a94f9398ee73be0fdf643de9b2/requirements.txt#L34

To prevent decoding errors on recently introduced types. In case you encounter decoding errors in the future this is something you could check.

[jadechip]

Thanks for your reply! Could you kindly elaborate a bit on what gets stored in Redis? I don't see any data being persisted🤔

[arjanz]

Basically I only stored the decoded metadata objects (because that is relatively expensive to decode) and the custom types, so you don't have to re-enter that every time.

[jadechip]

Thanks for clarifying, closing this issue for now. Cheers!