TypeError serializing a classmethod that refers to a module ContextVar (Pickle can) - Python 3.8

[LunarLanding]

testcontext.py

import contextvars
a = contextvars.ContextVar(__name__)
class A:
  @classmethod
  def g(cls):
    print(1)
  @classmethod
  def f(cls):
    print(a.get(None))

  def h(self):
    print(a.get(None))

#from testcontext import A; from cloudpickle import dumps; dumps(A.f)
#> TypeError

Python 3.8.3 | packaged by conda-forge | (default, Jun  1 2020, 17:43:00) 
[GCC 7.5.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from testcontext import A; from cloudpickle import dumps; dumps(A.f)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/ma/miniconda3/lib/python3.8/site-packages/cloudpickle/cloudpickle_fast.py", line 63, in dumps
    cp.dump(obj)
  File "/home/ma/miniconda3/lib/python3.8/site-packages/cloudpickle/cloudpickle_fast.py", line 548, in dump
    return Pickler.dump(self, obj)
TypeError: cannot pickle 'ContextVar' object
>>> from pickle import dumps
>>> dumps(A.f)
b'\x80\x04\x954\x00\x00\x00\x00\x00\x00\x00\x8c\x08builtins\x94\x8c\x07getattr\x94\x93\x94\x8c\x0btestcontext\x94\x8c\x01A\x94\x93\x94\x8c\x01f\x94\x86\x94R\x94.'
>>> 

[pierreglaser]

Hi. You are right that currently, cloudpickle does not explicitly provide support for pickling ContextVar. I personally haven't heard about the contextvars module, I'll take a look. Just to clarify, pickle cannot pickle ContextVars either:

In [1]: import contextvars

In [3]: c = contextvars.ContextVar(__name__)

In [4]: c
Out[4]: <ContextVar name='__main__' at 0x7f28b84e3c20>

In [5]: import pickle

In [6]: pickle.dumps(c)
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-6-4523793c002f> in <module>
----> 1 pickle.dumps(c)

TypeError: cannot pickle 'ContextVar' object

The reason why pickle is not erroring out when calling pickle.dumps(A.f) while as you correctly pointed out, A.f contains a reference to a ContextVar object, is that pickle serializes A.f through attribute access from its module. Essentially, pickle writes the code from A.__module__ import A; return getattr(A, 'f') to the pickle string, which does not requires pickling ContextVar. cloudpickle however, writes the necessary instructions to completely reconstruct the class, which includes code, and global variables referenced in the class, including the ContextVar variable here.

[LunarLanding]

Thanks for the quick reply. With my use-case, the contextvar is a module variable I serialize and de-serialize manually. I do this because the objects I am sending over depend on the contextvar being set to properly exist / being able to init/create, and their create method is sometimes async. I can for now use a proxy function that loads the contexvar and that cloudpickle does not look inside of. I think if this becomes the general case for objects that reference contextvars and are serialized, then it would make sense to make it the default for cloudpickle, but maybe it is too soon to say. I wonder if there is a way to indicate that via cloudpickle's API? I.e. set a default behavior for a type of object that normally returns a TypeError for serialization.