search

firehoseio / firehose

Build realtime Ruby web applications. Created by the fine folks at Poll Everywhere.
http://firehose.io/
MIT License
727 stars 72 forks source link

Calculate message buffer offsets in Redis (not Ruby) #55

Open bradgessler opened 8 years ago

bradgessler commented 8 years ago

Instead of grabbing all the messages from the message buffer and calculating offsets in Ruby, do it all on Redis.

The MessageBuffer class is a good place to start. https://github.com/firehoseio/firehose/blob/message-buffer/lib/firehose/server/message_buffer.rb. Be sure the test suite passes for whatever is implemented in place.

The message emitted from this new class should be MessageBuffer::Message objects. This will be compatible with all channel consumers.

Preferably this upgrade is done in a way that's compatible with live production upgrades (e.g. the Lua scripts we load for publishing shouldn't corrupt the message buffer arrays during a rolling restart)

Some work started for this at https://github.com/firehoseio/firehose/tree/message-buffer-redis. This branch may not be necessary depending on how we calculate the offsets for Redis.