tschellenbach / Stream-Framework

Stream Framework is a Python library, which allows you to build news feed, activity streams and notification systems using Cassandra and/or Redis. The authors of Stream-Framework also provide a cloud service for feed technology:
https://getstream.io/
Other
4.73k stars 541 forks source link

Concerns about a couple of use-cases #76

Open owais opened 10 years ago

owais commented 10 years ago

I'm thinking about using the lib or saas solution for my project and have a couple of questions. My main concerns are,

1) A multi-tenant architecture. I need the notification/activity key to be unique per sub-domain. How hard is it to change ID of the activity to inlcude the customer id or subdomain in addition to user id?

2) I also need per object counts. For example, I need to show unread messages in each discussion to a user. How hard is this to achieve with feedly?

owais commented 10 years ago

@tschellenbach thoughts?

tschellenbach commented 10 years ago

Hi Owais,

Good to hear from you. 1.) This is extremely easy using Stream. Simply initialize your feed with something like client.feed('feedname:subdomain-sep-userid') Also using Feedly you can probably just pass a string with the subdomain instead of just the user id. (Maybe you need to subclass and make a few small changes, but it wont be very hard)

2.) The notification feed stores the count of the total number of unread and unseen items using redis. It's available both in Stream Framework and when using the SAAS solution. We use Faye for realtime functionality. Setting that up yourself can be a bit tricky.

Could you tell us some more about what you're building?

Regards, Thierry

On Tue, Sep 30, 2014 at 11:49 AM, Owais Lone notifications@github.com wrote:

@tschellenbach https://github.com/tschellenbach thoughts?

— Reply to this email directly or view it on GitHub https://github.com/tschellenbach/Stream-Framework/issues/76#issuecomment-57290487 .

owais commented 10 years ago

@tschellenbach Thank you so much for the quick reply

I don't really need realtime functionality at this point. What I actually need with 2 is to be able to query the feed on some metadata. I don't need it to store denormalized counts on per object basis right now.

Lets say a user comments on a discussion and this generates some notifications. I want to be able to do something like this,

notifs = feed.filter(user_id=user.id, discussion_id=discussion.id)

or


A way to filter the feed by the type and id of the target object. Don't  need pre-stored counts right now.

The usecase is to implement a UI like this,

Discussion A (6)
Discussion B (2) Discussion C


where the numbers are the count of unread messages per user. I could implement this if I could do something like `len(feed.filter(user=user, discussion=discussion))`

The site is running really low number of users right now so denormalization is not a priority as of now.
tschellenbach commented 10 years ago

So you are implementing a notification feed, which shows the notifications like this:

Discussion A (6) Discussion B (2) Discussion C

You can specify an aggregation rule for the notification feed. In this case based on discussion. That would leave you with

Notification B (2 activities) Notification A (6 activities)

(sorted by last updated)

owais commented 10 years ago
from stream_framework.feeds.aggregated_feed.notification_feed import RedisNotificationFeed
from stream_framework.aggregators.base import BaseAggregator
from stream_framework.activity import Activity
from stream_framework.activity import AggregatedActivity
from stream_framework.verbs.base import Comment

class Event(Activity):
    pass

class AggregatedEvent(AggregatedActivity):
    pass

class CommentAggregator(BaseAggregator):
    '''
    Aggregates based on the same verb and same object
    '''
    def get_group(self, activity):
        verb = activity.verb.id
        object_id = activity.object_id
        content_type = activity.extra_context['content_type']
        group = '%s-%s-%s' % (verb, object_id, content_type)
        return group

    def rank(self, aggregated_activities):
        '''
        The ranking logic, for sorting aggregated activities
        '''
        aggregated_activities.sort(key=lambda a: a.updated_at, reverse=True)
        return aggregated_activities

class NotificationFeed(RedisNotificationFeed):
    key_format = 'feed:notifs:%(user_id)s'
    activity_class = Event
    aggregated_activity_class = AggregatedEvent
    aggregator_class = CommentAggregator

    def __init__(self, customer_id, user_id, **kwargs):
        user_id = '{}:{}'.format(customer_id, user_id)
        super(NotificationFeed, self).__init__(user_id, **kwargs)
# get feed for user
>> feed = NotificationFeed(user_id=40, customer_id=4)

# add event on discussion 30
>> feed.add(Event(40, Comment, 79, 30, datetime.datetime.now(), extra_context={'content_type': 8}))

# add 2 events on discussion 31
>> feed.add(Event(40, Comment, 79, 31, datetime.datetime.now(), extra_context={'content_type': 8}))
>> feed.add(Event(40, Comment, 79, 31, datetime.datetime.now(), extra_context={'content_type': 8}))

# fetch all activities
>> feed[:5]
[AggregatedActivity(2-79-8-commented) Actors 40: Objects [79],
 AggregatedActivity(2-79-9-commented) Actors 40: Objects [79]]

This gived me 2 aggregated activities for 2 discussions. Now I want to fetch the aggregated activity of a particular discussion. In the template, when I iterate over discussions and render each one, I want to be able to render it's unread message count also which is supplied from the aggregated activity. Can I do things like

feed.filter(object_id=XXX, content_type=YYY)

To directly get the activity of 1 particular discussion?

owais commented 10 years ago

@tschellenbach ^

tbarbugli commented 10 years ago

Hi Owais,

For performance reasons we use storages like Redis and Cassandra that requires data to be modelled around read paths; retrieve activities by something else than its ID is not an out-of-the box feature.

I think that the best way to do both things you are looking for is to store extra informations somewhere else than in the feed itself. For instance if you use Redis it is quite easy to build an index that you can query to retrieve activity ids by some activity attribute.

@tschellenbach https://github.com/tschellenbach do you have any good examples of how to do something like this? AFAIK we don't have an example app that does something like this.

Tommaso

2014-09-30 14:35 GMT+02:00 Owais Lone notifications@github.com:

@tschellenbach https://github.com/tschellenbach ^

— Reply to this email directly or view it on GitHub https://github.com/tschellenbach/Stream-Framework/issues/76#issuecomment-57306460 .