mailgun / expiringdict

Dictionary with auto-expiring values for caching purposes.
Apache License 2.0
344 stars 76 forks source link

ExpiringDict objects cant be pickled by Dill - but OrderedDict can be pickled !!! #35

Open bitranox opened 5 years ago

bitranox commented 5 years ago

In order to use that objects together with wrapt_timeout_decorator https://github.com/bitranox/wrapt_timeout_decorator

it needs to be pickable - but it is not. Ordered Dict can be pickled.

Test Code :

from expiringdict import ExpiringDict
import dill

try:
    from collections import OrderedDict
except ImportError:
    # Python < 2.7
    from ordereddict import OrderedDict

def test_pickle_ordered_dict():
    """
    >>> # test to pickle ordered dict
    >>> dict_test=OrderedDict()
    >>> dict_test['test'] = 1
    >>> pickled_object = dill.dumps(dict_test)
    >>> original_object = dill.loads(pickled_object)
    >>> original_object['test']
    1
    """
    pass

def test_pickle_expiring_dict():
    """
    >>> # test to pickle ordered dict
    >>> dict_test=ExpiringDict(max_len=200000, max_age_seconds=1800)
    >>> dict_test['test'] = 1
    >>> pickled_object = dill.dumps(dict_test)
    >>> original_object = dill.loads(pickled_object)  # doctest: +ELLIPSIS +NORMALIZE_WHITESPACE
    Traceback (most recent call last):
    ...
    TypeError: __init__() missing 2 required positional arguments: 'max_len' and 'max_age_seconds'

    """
    pass

def test_pickle_expiring_dict_pass():
    """
    >>> # test to pickle ordered dict
    >>> dict_test=ExpiringDict(max_len=200*1E3, max_age_seconds=30*60)
    >>> dict_test['test'] = 1
    >>> dict_test_pickable = OrderedDict(dict_test)
    >>> pickled_object = dill.dumps(dict_test_pickable)
    >>> original_object = dill.loads(pickled_object)  # doctest: +ELLIPSIS +NORMALIZE_WHITESPACE
    >>> original_object['test']
    1

    """
    pass

def main():
    pass

if __name__ == "__main__":
    main()
bitranox commented 5 years ago

I extended the Class to make it pickable and added some copy functions. Just look at https://github.com/mailgun/expiringdict/pull/36