taskforcesh / bullmq

BullMQ - Message Queue and Batch processing for NodeJS and Python based on Redis
https://bullmq.io
MIT License
5.83k stars 377 forks source link

GraphQL subscriptions #116

Open nicgene opened 4 years ago

nicgene commented 4 years ago

My stack uses graphql, graphql-redis-subscriptions (to implement pubsub) and BullMQ. I've discovered that I can use graphql-redis-subscriptions within my worker on completed event to update the subscription. Is there a better way to implement pubsub between BullMQ and other server-side API's or is that out of the scope of this project? Thanks for any info, this is my first message queue.

import { Job, Worker } from 'bullmq';
import { RedisPubSub } from 'graphql-redis-subscriptions';

...

const pubsub = new RedisPubSub({
  connection: {
    host: process.env.REDIS_HOST,
    port: Number(process.env.REDIS_PORT)
  }
});

const jobAdded = '';

worker.on('completed', (job: Job) => {
  ...
  let payload = {
    jobAdded: {
      jobId: job.id,
      jobName: job.name,
      jobData: job.data,
      jobProgress: job.progress
    }
  }

  pubsub.publish(jobAdded, payload);
  console.log(`${job.id} has completed!`);
});

...
manast commented 4 years ago

are you doing pubsub one-to-many or one-to-one? if you are doing one-to-one it is better to create another queue at the receiver of the messages, and have the publisher pushing the messages to the receiver queue, then you will never loose any messages in case of network or other issues.