Django and msgpack 0.6 can not serialize 'ExtType' object

Using the following code

import msgpack
import enum
from importlib import import_module

def default(obj):
    if issubclass(type(obj), enum.Enum):
        components = (obj.__module__, obj.__class__.__name__, obj.value)
        return msgpack.ExtType(42, msgpack.packb(components, use_bin_type=True))
    raise TypeError("Unknown type: %r" % (obj,))

def ext_hook(code, data):
    if code == 42:
        module, class_name, value = msgpack.unpackb(data, raw=False)
        if isinstance(module, bytes):
            module = module.decode()
        if isinstance(class_name, bytes):
            class_name = class_name.decode()
        obj = getattr(import_module(module), class_name)(value)
        return obj
    return msgpack.ExtType(code, data)

class Garbo(enum.Enum):
    thing = 1

This works fine in the shell:

In [2]: data = {'a': Garbo.thing}
   ...: packed = msgpack.packb(data, strict_types=True, default=default, use_bin_type=True)
   ...: unpacked = msgpack.unpackb(packed, ext_hook=ext_hook, raw=False)
   ...:
   ...: assert data == unpacked

In [3]:

In [3]: data
Out[3]: {'a': <Garbo.thing: 1>}

In [4]: unpacked
Out[4]: {'a': <Garbo.thing: 1>}

I wrote this on msgpack 0.5.6 but upgrading to 0.6.0 this works except inside my django process where it raises this:

In [3]: packed = msgpack.packb(data, strict_types=True, default=default, use_bin_type=True)
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-3-53de700462f7> in <module>
----> 1 packed = msgpack.packb(data, strict_types=True, default=default, use_bin_type=True)

/usr/local/lib/python3.7/site-packages/msgpack/__init__.py in packb(o, **kwargs)
     44     See :class:`Packer` for options.
     45     """
---> 46     return Packer(**kwargs).pack(o)
     47
     48

msgpack/_packer.pyx in msgpack._cmsgpack.Packer.pack()

msgpack/_packer.pyx in msgpack._cmsgpack.Packer.pack()

msgpack/_packer.pyx in msgpack._cmsgpack.Packer.pack()

msgpack/_packer.pyx in msgpack._cmsgpack.Packer._pack()

msgpack/_packer.pyx in msgpack._cmsgpack.Packer._pack()

TypeError: can not serialize 'ExtType' object

I know this is entirely something in my setup but I'm not sure the best way to debug this and was hoping you've seen something similar before?

Note: this does work with MSGPACK_PUREPYTHON=true which would be easier to debug if it didn't.

Since it's hitting this it means I have an issue with the switch statement: https://github.com/msgpack/msgpack-python/blob/v0.6.0/msgpack/_packer.pyx#L245-L279

Although this is true inside the shell where it's not working:

In [4]: type(msgpack.ExtType(code=42, data=b'1')) is msgpack.ExtType
Out[4]: True

I think it's caused by https://github.com/DataDog/dd-trace-py

msgpack / msgpack-python

Django and msgpack 0.6 can not serialize 'ExtType' object #366