dsc / bunch

A Bunch is a Python dictionary that provides attribute-style access (a la JavaScript objects).
http://github.com/dsc/bunch
MIT License
483 stars 174 forks source link

Bunch is Slow #31

Open jayd3e opened 9 years ago

jayd3e commented 9 years ago

I liked AttrDict. Then AttrDict was too slow. I liked Bunch. Then Bunch was too slow. These two libraries ended up consuming 1/2 and 1/3 of my request time when using them frequently, for what I had assumed was a very simple use-case. The main problem comes from two things 1) Bunch inherits from dict, so it brings along the entire dict implementation, which isn't always necessary(rarely is in fact). 2)bunchify aggressively goes through an entire nested dictionary, and puts copies of Bunch everywhere. This results in attributes that may never get called, getting implanted with the Bunch object.

Speeding this up proved to not take a lot of code. Check out this gist from @mmerickel, which I've copied below. It uses a light-weight DictProxy object to mediate access to the object, and it also lazy-loads each attribute. This decreased the amount of time that my app spend "Bunching" objects to relatively nothing, from 1/3 of my request time using Bunch master.

class DictProxy(object):
    """
    A proxy for a dictionary that allows attribute access to underlying keys.

    You may pass a custom ``wrapper`` to override the logic for wrapping
    various custom types.

    """
    def __init__(self, obj, wrapper=wrap):
        self.obj = obj

    def __getitem__(self, key):
        return self.wrapper(self.obj[key])

    def __getattr__(self, key):
        try:
            return self.wrapper(getattr(self.obj, key))
        except AttributeError:
            try:
                return self[key]
            except KeyError:
                raise AttributeError(key)

    # you probably also want to proxy important list properties along like
    # items(), iteritems() and __len__

class ListProxy(object):
    """
    A proxy for a list that allows for wrapping items.

    You may pass a custom ``wrapper`` to override the logic for wrapping
    various custom types.

    """
    def __init__(self, obj, wrapper=wrap):
        self.obj = obj

    def __getitem__(self, key):
        return self.wrapper(self.obj[key])

    # you probably also want to proxy important list properties along like
    # __iter__ and __len__

def wrap(value):
    """
    The top-level API for wrapping an arbitrary object.

    This only works for ``dict``, ``list`` and ``tuple`` types. If you want
    to wrap other types you may write your own wrap and pass ``wrapper=`` to
    ``DictProxy`` and ``ListProxy``.

    """
    if isinstance(value, dict):
        return DictProxy(value)
    if isinstance(value, (tuple, list)):
        return ListProxy(value)
    return value
jkomusin commented 8 years ago

You might try moving this to the munch project, which seems more lively: https://github.com/Infinidat/munch