turicas / rows

A common, beautiful interface to tabular data, no matter the format
GNU Lesser General Public License v3.0
870 stars 134 forks source link

Make Row object pickable #349

Open berinhard opened 4 years ago

berinhard commented 4 years ago

I was trying to cache rows.table.Row object in a Django project using django-redis but I ran into a serialization error with pickle. Here's the traceback beginning from the relevant entry point, a cache.set call:

~/.virtualenvs/brasil.io/lib/python3.6/site-packages/cache_memoize/__init__.py in inner(*args, **kwargs)
    123                     cache.set(cache_key, True, timeout)
    124                 else:
--> 125                     cache.set(cache_key, result, timeout)
    126                 if miss_callable:
    127                     miss_callable(*args, **kwargs)

~/.virtualenvs/brasil.io/lib/python3.6/site-packages/django_redis/cache.py in _decorator(self, *args, **kwargs)
     30     def _decorator(self, *args, **kwargs):
     31         try:
---> 32             return method(self, *args, **kwargs)
     33         except ConnectionInterrupted as e:
     34             if self._ignore_exceptions:

~/.virtualenvs/brasil.io/lib/python3.6/site-packages/django_redis/cache.py in set(self, *args, **kwargs)
     65     @omit_exception
     66     def set(self, *args, **kwargs):
---> 67         return self.client.set(*args, **kwargs)
     68 
     69     @omit_exception

~/.virtualenvs/brasil.io/lib/python3.6/site-packages/django_redis/client/default.py in set(self, key, value, timeout, version, client, nx, xx)
    109         """
    110         nkey = self.make_key(key, version=version)
--> 111         nvalue = self.encode(value)
    112 
    113         if timeout is DEFAULT_TIMEOUT:

~/.virtualenvs/brasil.io/lib/python3.6/site-packages/django_redis/client/default.py in encode(self, value)
    321 
    322         if isinstance(value, bool) or not isinstance(value, int):
--> 323             value = self._serializer.dumps(value)
    324             value = self._compressor.compress(value)
    325             return value

~/.virtualenvs/brasil.io/lib/python3.6/site-packages/django_redis/serializers/pickle.py in dumps(self, value)
     19 
     20     def dumps(self, value):
---> 21         return pickle.dumps(value, self._pickle_version)
     22 
     23     def loads(self, value):

PicklingError: Can't pickle <class 'rows.table.Row'>: attribute lookup Row on rows.table failed

It would be nice to have this object as a pickleable one so we could cache it without manual parsing.