search

fanout / django-eventstream

Server-Sent Events for Django
MIT License
650 stars 85 forks source link

Sending a message from a celery worker #34

Closed TheReal1604 closed 5 years ago

TheReal1604 commented 5 years ago

hello!

maybe one of you got an idea how to resolve this issue.

I try to send a message to the user eventstream via a django signal - as long as the signal is called from the main django process the message is send to the client. Unfortunately some background jobs should also use this signal. Its triggering correctly in the celery worker, but the send message is never received by the client.

Maybe someone got a hint in the right direction?

Thank you!

svvitale commented 5 years ago

Since the event stream subscribers are connecting to the main Django process, that's the only place where you can broadcast back out. In the past, I've created a Django endpoint and did a POST back from the celery worker to trigger the broadcast. That won't scale very well at all, but it's a handy workaround for small projects.

For a broader, production-level solution, you'll want to either set up your own pushpin proxy server or go with a managed solution like Fanout Cloud.

TheReal1604 commented 5 years ago

Thanks for the idea, it’s a little project, so the workaround should be good enough.

svvitale commented 5 years ago

Fair warning, this also breaks as soon as you move to multiple django instances.

TheReal1604 commented 5 years ago

Close this issue, as it is answered. Thanks @svvitale !

ivan-tomicic commented 1 year ago

Since the event stream subscribers are connecting to the main Django process, that's the only place where you can broadcast back out. In the past, I've created a Django endpoint and did a POST back from the celery worker to trigger the broadcast. That won't scale very well at all, but it's a handy workaround for small projects.

For a broader, production-level solution, you'll want to either set up your own pushpin proxy server or go with a managed solution like Fanout Cloud.

Hi, I am currently running into the same problem as @TheReal1604, I have a React frontend app which communicates with a backend app written in Django. In Django I create tasks with Celery and want to notify the user's browser whether the task finished successfully or not. I tried it with SSE (Server-Sent Events) but the issue is the same as his, I can't broadcast it out from the celery task. I installed pushpin and tested it, but my question is how does pushpin fit into the solution? Do I broadcast the task's success/failure from celery to my backend via pushpin, or from celery to frontend via pushpin? Sorry if I don't make any sense, I am still a junior :/

jkarneges commented 1 year ago

Hi @ivan-tomicic, pushpin maintains the client connections on behalf of your backend app. Your celery tasks can/should publish messages directly to pushpin. Django-eventstream already knows how to do this. Ensure settings.GRIP_URL is set to a value pointing at pushpin, and then call send_event as usual.