taskforcesh / bullmq

BullMQ - Message Queue and Batch processing for NodeJS and Python based on Redis
https://bullmq.io
MIT License
6.01k stars 387 forks source link

[Feature request] search a specific job #2135

Open felixmosh opened 1 year ago

felixmosh commented 1 year ago

Is your feature request related to a problem? Please describe. It will be great if queue can search a job with specific name, id, data etc...

Describe the solution you'd like

   const jobs = await queue.search({jobData: 'contains some data'});

Describe alternatives you've considered Searching jobs on the redis it self

ccollie commented 1 year ago

So I have (working) code in a personal project that does something similar. We can do something like:

export interface FindJobsByFilterOptions {
  filter?: string;
  cursor?: string;
  status?: JobSearchStatus;
  pattern?: string;
  count?: number;
}

export interface FilteredJobsResult {
  cursor: string;
  total: number;
  current: number;
  jobs: Job[];
}

const filter = 'job.data.action == "create" && job.data.widget = "sprocket" && job.processTime < 100';

let { jobs, total, current, cursor } = await findJobsByFilter(queue, 
    { 
        filter, count: 10, cursor: null, pattern: 'job-*', status: "completed" 
    } 
)
if (jobs.length) {
    do {
        console.log(jobs);
        {jobs, cursor} = await findJobsByFilter(queue, { cursor } ); 
    } while(cursor)
}        

Currently it uses jsep to parse js compatible expressions, determines which fields need to be fetched from redis and tries to as efficiently as possible retrieve the minimum data necessary to satisfy the predicate.

Notice that it uses a cursor based approach to reduce the load on redis and provide a paging interface. Im not wedded to the expression syntax, and previous iterations used a mongo styled query document.

If Manuel thinks this fits within the philosophy of bullmq, I can clean it up and create a PR. Otherwise feel free to grab the code and plug away.

kilobyte2007 commented 5 months ago

Are there any plans to develop this feature?

felixmosh commented 5 months ago

Once the underline lib (BullX) will add support for it.

For a meanwhile, you can up vote https://github.com/taskforcesh/bullmq/issues/2135

kilobyte2007 commented 4 months ago

Once the underline lib (BullX) will add support for it.

For a meanwhile, you can up vote #2135

Isn't this the underline lib?

felixmosh commented 4 months ago

By "underline lib" I mean bull / bullmq which is used by the board to access all the data.

kilobyte2007 commented 4 months ago

By "underline lib" I mean bull / bullmq which is used by the board to access all the data.

I get it, but this issue is open on the bullmq repo so I thought that was given :D

leedia-tech commented 4 months ago

Any news on this?

bfbechlin commented 3 months ago

This feature is really important. It would be amazing if it were prioritized 🙂

ts-shivam commented 2 months ago

A must have feature missing.

pedropuppim commented 2 months ago

This feature is much needed.