search

uber-archive / cherami-server

Distributed, scalable, durable, and highly available message queue system. This project is deprecated and not maintained.
https://eng.uber.com/cherami/
MIT License
1.42k stars 102 forks source link

Outputhost: add support for delay and skip-older #234

Closed kirg closed 7 years ago

coveralls commented 7 years ago

Coverage Status

Coverage decreased (-0.03%) to 68.492% when pulling 3517106ba62c36a494cdb3ce41ab34aec819673d on outputhost-delay-and-skipolder into ec8103f2637ad4749c8880d88fb56b8c953a8af6 on master.

coveralls commented 7 years ago

Coverage Status

Coverage increased (+0.06%) to 68.581% when pulling 3d1396738c53919a6d664e7a8e1286ed9c3233b6 on outputhost-delay-and-skipolder into ec8103f2637ad4749c8880d88fb56b8c953a8af6 on master.

kirg commented 7 years ago

There's some missing stuff here. One thing is that we are preventing messages from getting into message cache, but we really ought to expire messages in the cache as well (by acking them). Some logic in messagecache.go would solve this. If you pursue that, be sure that you appropriately expire the smart-retry LKG message.

The 'skip-older' is primarily intended as a mechanism to flush out the backlog quickly. And what you suggest does help enforce 'skip-older' more strictly .. but I am going to leave this out for now; we can perhaps consider this in the future if we find that there are too many messages hanging around in the message cache for unacceptably long.