Closed andrenro closed 10 years ago
Hi !
The global name is used for the storing the activity. (mapping the id to the activity) (For which we should probably really find a way to make it smaller) (@tbarbugli, ideas on this?)
So its normal for data to end up in global. From your setup i would expect: 1 in feed:user 1 in global number of followers in the feed:normal
The feedly setup looks ok. I see two possible reasons why there is nothing in feed:normal
A.) the list of follower ids returned is empty B.) the celery tasks generated by create fanout tasks don't get processed
(You could try enabling CELERY_ALWAYS_EAGER = True CELERY_EAGER_PROPAGATES_EXCEPTIONS = True) Dont use these settings in production though!
What sort of application are you building?
Best of luck!
Hi!:)
Im building a social gift-planner/wishlist-app! Im trying to integrate feedly so that I can make REST-based feeds on the form "User1 added product1 to wishlist1" :) So what i would need is that other users (User1's friends) can receive this kind of feed. I would guess possibility B) , but I've tried with those settings, nothing happened..Any other possible solutions?
Current settings.py: FEEDLY_NYDUS_CONFIG = { 'CONNECTIONS': { 'redis': { 'engine': 'nydus.db.backends.redis.Redis', 'router': 'nydus.db.routers.redis.PrefixPartitionRouter', 'hosts': { 0: {'prefix': 'default', 'db': 0, 'host': '127.0.0.1', 'port': 6379}, } }, } }
BROKER_URL = 'redis://127.0.0.1:6379/0' CELERY_RESULT_BACKEND = 'redis://localhost:6379/0' CELERY_ALWAYS_EAGER = True CELERY_EAGER_PROPAGATES_EXCEPTIONS = True
I will check get_user_follower_ids() for emptiness (I am pretty certain that the friend-relationships exists in the database though). I would need something that works in production also :)
Sounds cool,
Celery is a great for running async tasks, but it can be a bit of a learning curve. The docs are here: http://www.celeryproject.org/
Try to verify that the code reaches UserWishFeedly.fanout Could you dump the local variables you get when going into UserWishFeedly.fanout?
I think there might be a problem with my redis setup as well, I've installed rabbitMQ and Redis but it seems that the settings for redis/celery does nothing . If i stop the rabbitMQ (rabbitmqctl stop) i will get an "error [111] connection refused" when I try to add something to the feeds. My impression was that Redis was now my backend, and that I do not need to have rabbitMQ server running? Im confused.. Celery uses EITHER rabbitMQ or Redis as its task broker, right?
On Wed, Dec 18, 2013 at 12:33 PM, Thierry Schellenbach < notifications@github.com> wrote:
Sounds cool,
Celery is a great for running async tasks, but it can be a bit of a learning curve. The docs are here: http://www.celeryproject.org/
Try to verify that the code reaches UserWishFeedly.fanout Could you dump the local variables you get when going into UserWishFeedly.fanout?
— Reply to this email directly or view it on GitHubhttps://github.com/tschellenbach/Feedly/issues/16#issuecomment-30834733 .
Mvh Andreas Røed
when CELERY_ALWAYS_EAGER is set to True tasks are executed in the same thread and celery does not connect to the task broker. I think that your celery settings are ignored / unused; you can find docs about this here https://docs.celeryproject.org/en/latest/django/first-steps-with-django.html
2013/12/18 Andreas Røed notifications@github.com
I think there might be a problem with my redis setup as well, I've installed rabbitMQ and Redis but it seems that the settings for redis/celery does nothing . If i stop the rabbitMQ (rabbitmqctl stop) i will get an "error [111] connection refused" when I try to add something to the feeds. My impression was that Redis was now my backend, and that I do not need to have rabbitMQ server running? Im confused.. Celery uses EITHER rabbitMQ or Redis as its task broker, right?
On Wed, Dec 18, 2013 at 12:33 PM, Thierry Schellenbach < notifications@github.com> wrote:
Sounds cool,
Celery is a great for running async tasks, but it can be a bit of a learning curve. The docs are here: http://www.celeryproject.org/
Try to verify that the code reaches UserWishFeedly.fanout Could you dump the local variables you get when going into UserWishFeedly.fanout?
— Reply to this email directly or view it on GitHub< https://github.com/tschellenbach/Feedly/issues/16#issuecomment-30834733> .
Mvh Andreas Røed
— Reply to this email directly or view it on GitHubhttps://github.com/tschellenbach/Feedly/issues/16#issuecomment-30837484 .
So I got it working correctly when using CELERY_ALWAYS_EAGER = True ! :) Only thing left is to make the broker settings work correctly! I don't get any errors when CELERY_ALWAYS_EAGER = False, it just doesn't fetch the newest feeds properly..
On Wed, Dec 18, 2013 at 1:49 PM, Tommaso Barbugli notifications@github.comwrote:
when CELERY_ALWAYS_EAGER is set to True tasks are executed in the same thread and celery does not connect to the task broker. I think that your celery settings are ignored / unused; you can find docs about this here
https://docs.celeryproject.org/en/latest/django/first-steps-with-django.html
2013/12/18 Andreas Røed notifications@github.com
I think there might be a problem with my redis setup as well, I've installed rabbitMQ and Redis but it seems that the settings for redis/celery does nothing . If i stop the rabbitMQ (rabbitmqctl stop) i will get an "error [111] connection refused" when I try to add something to the feeds. My impression was that Redis was now my backend, and that I do not need to have rabbitMQ server running? Im confused.. Celery uses EITHER rabbitMQ or Redis as its task broker, right?
On Wed, Dec 18, 2013 at 12:33 PM, Thierry Schellenbach < notifications@github.com> wrote:
Sounds cool,
Celery is a great for running async tasks, but it can be a bit of a learning curve. The docs are here: http://www.celeryproject.org/
Try to verify that the code reaches UserWishFeedly.fanout Could you dump the local variables you get when going into UserWishFeedly.fanout?
— Reply to this email directly or view it on GitHub< https://github.com/tschellenbach/Feedly/issues/16#issuecomment-30834733>
.
Mvh Andreas Røed
— Reply to this email directly or view it on GitHub< https://github.com/tschellenbach/Feedly/issues/16#issuecomment-30837484> .
— Reply to this email directly or view it on GitHubhttps://github.com/tschellenbach/Feedly/issues/16#issuecomment-30838903 .
Mvh Andreas Røed
Thats because tasks are waiting in the broker to be consumed by a celery worker. You need to start a celery worker for that ;) On 18 Dec 2013 14:24, "Andreas Røed" notifications@github.com wrote:
So I got it working correctly when using CELERY_ALWAYS_EAGER = True ! :) Only thing left is to make the broker settings work correctly! I don't get any errors when CELERY_ALWAYS_EAGER = False, it just doesn't fetch the newest feeds properly..
On Wed, Dec 18, 2013 at 1:49 PM, Tommaso Barbugli notifications@github.comwrote:
when CELERY_ALWAYS_EAGER is set to True tasks are executed in the same thread and celery does not connect to the task broker. I think that your celery settings are ignored / unused; you can find docs about this here
https://docs.celeryproject.org/en/latest/django/first-steps-with-django.html
2013/12/18 Andreas Røed notifications@github.com
I think there might be a problem with my redis setup as well, I've installed rabbitMQ and Redis but it seems that the settings for redis/celery does nothing . If i stop the rabbitMQ (rabbitmqctl stop) i will get an "error [111] connection refused" when I try to add something to the feeds. My impression was that Redis was now my backend, and that I do not need to have rabbitMQ server running? Im confused.. Celery uses EITHER rabbitMQ or Redis as its task broker, right?
On Wed, Dec 18, 2013 at 12:33 PM, Thierry Schellenbach < notifications@github.com> wrote:
Sounds cool,
Celery is a great for running async tasks, but it can be a bit of a learning curve. The docs are here: http://www.celeryproject.org/
Try to verify that the code reaches UserWishFeedly.fanout Could you dump the local variables you get when going into UserWishFeedly.fanout?
— Reply to this email directly or view it on GitHub<
https://github.com/tschellenbach/Feedly/issues/16#issuecomment-30834733>
.
Mvh Andreas Røed
— Reply to this email directly or view it on GitHub< https://github.com/tschellenbach/Feedly/issues/16#issuecomment-30837484>
.
— Reply to this email directly or view it on GitHub< https://github.com/tschellenbach/Feedly/issues/16#issuecomment-30838903> .
Mvh Andreas Røed
— Reply to this email directly or view it on GitHubhttps://github.com/tschellenbach/Feedly/issues/16#issuecomment-30840972 .
So this should be closed?
Hi! I manage to get the get_user_feed() work properly, but when I try to fetch followers feeds, something strange happens. I think it might be an issue connected to fanout/insertion into feedly: Ive used the pin_feedly example as a starting point.
Here is some code: models.py:
Test implementation of feedly:
class WishFeed(RedisFeed): key_format = 'feed:normal:%(user_id)s'
class UserWishFeed(WishFeed): key_format = 'feed:user:%(user_id)s'
class UserWishFeedly(Feedly): feed_classes = dict( normal=WishFeed, )
views.py @csrf_exempt @api_view(['GET']) @login_required def friends_wish_feed(request, _args, *_kwargs): feed = feedly.get_feeds(request.user.id)['normal'] act_list = [] json_data = [] for acts in feed[:25]: act_list.append(acts) json_data = json.dumps(act_list) return Response(json_data)
this is how I insert into the feeds: views.py ... activity = Activity(wishlist.user,WishVerb,in_prod.id) feed = UserWishFeed(wishlist.user.id) feed.add(activity) feedly.add_wish(wishlist.user, activity) ...
from redis-cli after insertion i will get this: 127.0.0.1:6379> keys * 1) "global:3" 2) "global:4" 3) "feed:user:7" 4) "global:7" 5) "global:9"
if i query 3) : 127.0.0.1:6379> zrange "feed:user:7" 0 1 1) "13873587370000000000003005" 2) "13873587410000000000001005"
127.0.0.1:6379> hgetall "global:3" 1) "13873587470000000000006005" 2) "7,5,6,0,1387358747,"
Seems like the insertion went fine, except that it did not define the correct key_format? How can I fix this?
Why does it say "global:" instead of "user:normal:" like I defined in WishFeed(RedisFeed) ?
The function friends_wish_feed() returns [] every time..
Help appreciated!! :)