`TypedDict` is not serializable

[mtitov]

traceback from the original issue

    return base64.b64encode(dill.dumps(obj)).decode('utf-8')
  File "/autofs/nccs-svm1_home1/sharplo/venv_rct/lib/python3.9/site-packages/dill/_dill.py", line 263, in dumps
    dump(obj, file, protocol, byref, fmode, recurse, **kwds)#, strictio)
  File "/autofs/nccs-svm1_home1/sharplo/venv_rct/lib/python3.9/site-packages/dill/_dill.py", line 235, in dump
    Pickler(file, protocol, **_kwds).dump(obj)
  File "/autofs/nccs-svm1_home1/sharplo/venv_rct/lib/python3.9/site-packages/dill/_dill.py", line 394, in dump
    StockPickler.dump(self, obj)
  File "/opt/cray/pe/python/3.9.12.1/lib/python3.9/pickle.py", line 487, in dump
    self.save(obj)
  File "/autofs/nccs-svm1_home1/sharplo/venv_rct/lib/python3.9/site-packages/dill/_dill.py", line 388, in save
    StockPickler.save(self, obj, save_persistent_id)
  File "/opt/cray/pe/python/3.9.12.1/lib/python3.9/pickle.py", line 603, in save
    self.save_reduce(obj=obj, *rv)
  File "/opt/cray/pe/python/3.9.12.1/lib/python3.9/pickle.py", line 717, in save_reduce
    save(state)
  File "/autofs/nccs-svm1_home1/sharplo/venv_rct/lib/python3.9/site-packages/dill/_dill.py", line 388, in save
    StockPickler.save(self, obj, save_persistent_id)
  File "/opt/cray/pe/python/3.9.12.1/lib/python3.9/pickle.py", line 560, in save
    f(self, obj)  # Call unbound method with explicit self
  File "/autofs/nccs-svm1_home1/sharplo/venv_rct/lib/python3.9/site-packages/dill/_dill.py", line 1186, in save_module_dict
    StockPickler.save_dict(pickler, obj)
  File "/opt/cray/pe/python/3.9.12.1/lib/python3.9/pickle.py", line 971, in save_dict
    self._batch_setitems(obj.items())
  File "/opt/cray/pe/python/3.9.12.1/lib/python3.9/pickle.py", line 997, in _batch_setitems
    save(v)
  File "/autofs/nccs-svm1_home1/sharplo/venv_rct/lib/python3.9/site-packages/dill/_dill.py", line 388, in save
    StockPickler.save(self, obj, save_persistent_id)
  File "/opt/cray/pe/python/3.9.12.1/lib/python3.9/pickle.py", line 560, in save
    f(self, obj)  # Call unbound method with explicit self
  File "/opt/cray/pe/python/3.9.12.1/lib/python3.9/pickle.py", line 931, in save_list
    self._batch_appends(obj)
  File "/opt/cray/pe/python/3.9.12.1/lib/python3.9/pickle.py", line 955, in _batch_appends
    save(x)
  File "/autofs/nccs-svm1_home1/sharplo/venv_rct/lib/python3.9/site-packages/dill/_dill.py", line 388, in save
    StockPickler.save(self, obj, save_persistent_id)
  File "/opt/cray/pe/python/3.9.12.1/lib/python3.9/pickle.py", line 603, in save
    self.save_reduce(obj=obj, *rv)
  File "/opt/cray/pe/python/3.9.12.1/lib/python3.9/pickle.py", line 717, in save_reduce
    save(state)
  File "/autofs/nccs-svm1_home1/sharplo/venv_rct/lib/python3.9/site-packages/dill/_dill.py", line 388, in save
    StockPickler.save(self, obj, save_persistent_id)
  File "/opt/cray/pe/python/3.9.12.1/lib/python3.9/pickle.py", line 560, in save
    f(self, obj)  # Call unbound method with explicit self
  File "/autofs/nccs-svm1_home1/sharplo/venv_rct/lib/python3.9/site-packages/dill/_dill.py", line 1186, in save_module_dict
    StockPickler.save_dict(pickler, obj)
  File "/opt/cray/pe/python/3.9.12.1/lib/python3.9/pickle.py", line 971, in save_dict
    self._batch_setitems(obj.items())
  File "/opt/cray/pe/python/3.9.12.1/lib/python3.9/pickle.py", line 997, in _batch_setitems
    save(v)
  File "/autofs/nccs-svm1_home1/sharplo/venv_rct/lib/python3.9/site-packages/dill/_dill.py", line 388, in save
    StockPickler.save(self, obj, save_persistent_id)
  File "/opt/cray/pe/python/3.9.12.1/lib/python3.9/pickle.py", line 603, in save
    self.save_reduce(obj=obj, *rv)
  File "/opt/cray/pe/python/3.9.12.1/lib/python3.9/pickle.py", line 717, in save_reduce
    save(state)
  File "/autofs/nccs-svm1_home1/sharplo/venv_rct/lib/python3.9/site-packages/dill/_dill.py", line 388, in save
    StockPickler.save(self, obj, save_persistent_id)
  File "/opt/cray/pe/python/3.9.12.1/lib/python3.9/pickle.py", line 560, in save
    f(self, obj)  # Call unbound method with explicit self
  File "/autofs/nccs-svm1_home1/sharplo/venv_rct/lib/python3.9/site-packages/dill/_dill.py", line 1186, in save_module_dict
    StockPickler.save_dict(pickler, obj)
  File "/opt/cray/pe/python/3.9.12.1/lib/python3.9/pickle.py", line 971, in save_dict
    self._batch_setitems(obj.items())
  File "/opt/cray/pe/python/3.9.12.1/lib/python3.9/pickle.py", line 997, in _batch_setitems
    save(v)
  File "/autofs/nccs-svm1_home1/sharplo/venv_rct/lib/python3.9/site-packages/dill/_dill.py", line 388, in save
    StockPickler.save(self, obj, save_persistent_id)
  File "/opt/cray/pe/python/3.9.12.1/lib/python3.9/pickle.py", line 578, in save
    rv = reduce(self.proto)
  File "/autofs/nccs-svm1_home1/sharplo/venv_rct/lib/python3.9/site-packages/radical/utils/typeddict.py", line 270, in __getattr__
    else            : return data[k]
KeyError: '__getstate__'

[mtitov]

@andre-merzky this issue is about that corresponding method __getstate__ is not picked from the base class dict.

When TypedDict was introduced it wasn't inherited from any class, thus it had (and currently has) the following __getattr__ method https://github.com/radical-cybertools/radical.utils/blob/69d2a2266226bde869909d197a401393c94c91a7/src/radical/utils/typeddict.py#L263-L270

But original Munch was inherited from DictMixin and thus had __getattr__ method https://github.com/radical-cybertools/radical.utils/blob/d00925de1e569121cc82c5029f0d05b8536994bd/src/radical/utils/munch.py#L155-L165

After we made TypedDict inherited from dict, method __getattr__ wasn't updated should we just add

        if k.startswith('__'):
            return object.__getattribute__(self, k)

?

[andre-merzky]

Yep, thanks - added that. A special handling for _data was needed also though - please have a look if that works for you (#378)