Feature Request: Search

[JoJ123]

It would be nice to search for elements in the queue on the Arena Board.

Do you had that feature already in mind?

[ccollie]

What fields should be available to search ? Would this include the job data ?

[rosslavery]

This is a big one for us. We process ~ 1 million jobs daily and being able to verify if a job ran, when it ran, etc. would be great.

We'd need to search by: job name, job data.

[ccollie]

I don't have a solution (yet), but I can help if you need something in the shorter term.

i did something roughly related for another project: redis-timeseries-stream, and I have a similar filtering mechanism in mind for a bull project I'm working on.

[ccollie]

@rosslavery how would that verification process look ? I guess there would be a dialog where you could look up a job by Id ?

[rronalddas]

@rosslavery did you try using taskforce from the OptimalBull Team? They have a search functionality

[JoJ123]

Hi, thx for all the answers so far. For my project the best would be to have a regex search on the job data

[ccollie]

With your volume of jobs the most efficient route would be something written in lua. Keep in mind that lua does'nt use posix regexes, so there may be compromises depending on the complexity of your searches.

Do you have an example of the data you're searching ?

[JoJ123]

{ "content": { "machineName": "TestMachine", "machinePath": "123e4567e891012ed68aae95ed\\applications", "applicationName": "TestOffline", "severity": "Critical", "comment": "Critical", "status": "offline", "timestamp": "12/11/2020 19:49:10" }, "notificationId": "11111c2222223ac3333336af", "userId": "11111c2222223ac3333336af" }

This would be one example for one of our queues the content is not more than that.

[ccollie]

Final question (maybe). What's an example search you'd like to run against this ? Feel free to express it in js or as a mongo style query.

[JoJ123]

In this content case the best would be to filter for this field notificationId, but the content can change on different queues.

So here I would like to have a "search bar" like in Kibana / Logz.io and such type inside "11111c2222223ac3333336af" to get all matching jobs.

I'm open to more questions 👍

[ccollie]

I see. That's much more straightforward than what I'm working on, but I'll go ahead and support a simple text search over job.data. Unfortunately I'm not sure how much bandwidth I'll have on the UI side, so if someone could help there i'd appreciate it.

[mannharleen]

Any update or progress ? This feature is really important.

[ccollie]

I wrote this specifically for a monitoring library I'm writing specifically for bullmq, so it may not work for bull. Essentially it implements a mongo compatible query library in lua that does a filter over job data in redis. There are tests included for most of the functionality included, though more are needed (see here)

It can probably be extracted for use in bull-board, but my bandwidth is limited. If someone wants to tackle it, I'd be happy to pitch in.

[theoilie]

Bumping this one 🙂 is there a simple version of search that could be implemented -- for example, filtering by one JSON field in data or returnValue within a single queue, or a simple string#includes() for an entire job's stringified data?

[luco]

Bump on this.

[idbenami]

Bump again 🙃 search by a field in the data or returnValue sounds good to me.

In my project we're working with multi-tenancy, so I'm adding clientId field to the job's data, would be really nice to filter by that field.

[srinich-dev]

I have the same use case. Please prioritise it.

[Kingkross1]

Search Feature in bullboard , would be great !