search

felixmosh / bull-board

🎯 Queue background jobs inspector
MIT License
2.36k stars 366 forks source link

Using Empty option to clean 7 million tasks crash redis #845

Open or-opus opened 2 weeks ago

or-opus commented 2 weeks ago

Hi We are using BullBoard with bullMq which is a life saver.

We paused a queue for couple of hours and during that time it got full with 7 million tasks (in paused), so we decided to clean those 7m so we wont have to process them.

Once clicking on Empty, our ElasticChace (AWS redis solution) instance got chocked,.all our services where not able to communicate with it, neither redis insight UI as it claims that a blocking script is running.

After 10 min it came back to life, after our entire data got.lost/deleted (assuming the EC instance was crashed)

Any idea what can cause that? Any scale limitations you are aware of?

Thx!

felixmosh commented 2 weeks ago

Currently there is no limit of how many items to clean (it uses BullMQ directly, with max_safe_integer as the clean limit) Meaning that there is no any logic on the bull-board side ... I'm not aware of a scale limit of BullMQ itself, maybe you should ask there?