seomoz / qless-core

Core Lua Scripts for qless
MIT License
85 stars 33 forks source link

Consider truncating history as it grows #25

Closed myronmarston closed 11 years ago

myronmarston commented 11 years ago

We have some old jobs that have been bouncing around our staging environment for a while, and we've retried them repeatedly with larger sets of failed jobs. As a result, they've got over 17K events in their history. This causes memory bloat -- the history for this job is using up ~ 2 MB of redis memory.

While you can argue that we shouldn't keep retrying the job (and you'd be right) it would also be nice if Qless protected against the history becoming too bloated. What do you think about this?

dlecocq commented 11 years ago

I dig it. Let me take a quick pass at it.

myronmarston commented 11 years ago

Closed by #26.