Stream Framework is a Python library, which allows you to build news feed, activity streams and notification systems using Cassandra and/or Redis. The authors of Stream-Framework also provide a cloud service for feed technology:
cPickle is significantly faster than the pure-python pickle module:
In [2]: obj = {'a':'b'}
In [5]: %timeit cPickle.dumps(obj)
1000000 loops, best of 3: 1.29 µs per loop
In [6]: %timeit pickle.dumps(obj)
100000 loops, best of 3: 12.8 µs per loop
In [7]: serialized = cPickle.dumps(obj)
In [9]: %timeit cPickle.loads(serialized)
100000 loops, best of 3: 1.39 µs per loop
In [10]: %timeit pickle.loads(serialized)
100000 loops, best of 3: 14.7 µs per loop
With this simple object the performance is a factor of 10 better. The disadvantage of using cPickle is you cannot create custom pickle subclasses, but seeing as we do not do this and there is no option to do this it is safe to import cPickle over pickle, where available. This should lead to a nice speed increase when serializing and deserializing complex activity streams.
cPickle is significantly faster than the pure-python pickle module:
With this simple object the performance is a factor of 10 better. The disadvantage of using cPickle is you cannot create custom pickle subclasses, but seeing as we do not do this and there is no option to do this it is safe to import cPickle over pickle, where available. This should lead to a nice speed increase when serializing and deserializing complex activity streams.