search

hubotio / hubot-redis-brain

A hubot script to persist hubot's brain using redis
69 stars 55 forks source link

Hash versus giant string? #64

Open johnseekins-pathccm opened 8 months ago

johnseekins-pathccm commented 8 months ago

Someone has written an MIT-licensed Redis brain that uses hget/hset commands for adding removing data from the brain. Is it worth exploring this more broadly for the main Redis brain? I think it would make a horizontally scalable Hubot backed by Redis possible?

joeyguerra commented 8 months ago

Yes. I think so because this one just stores everything in a single JSON document and saves that document to Redis. That large granularity has scaling and usage limits.

@johnseekins-pathccm do you have a current use case? Or were you just thinking about possible use cases for the future?

johnseekins-pathccm commented 8 months ago

I mean...I'd like our deploys of Hubot to not require downtime. Which using this brain does require. Also, I think it would address issues like #39 .

joeyguerra commented 8 months ago

Can you elaborate on what's causing the downtime?

johnseekins-pathccm commented 8 months ago

My understanding is we can only have one instance of Hubot running at a time (because the brain reads/writes a string, which can overwrite values between two instances). Thus, we tear down the old instance before starting the new one.

joeyguerra commented 8 months ago

I'm speculating here ... I think the problem with > 1 instances of Hubot is that in the "save to Redis" scenario, the last write wins. Which could result in data loss, multiple instances writing at the same time. Both of the hubot-redis implementations have this problem. I think it's a design problem with the Hubot Brain. It assumes that there's just 1 instance of Hubot and that all data is kept in memory. I believe that was the intent.

johnseekins-pathccm commented 8 months ago

In that case, I guess we leave this be as there's no real difference. Other than some scenarios have slightly improved read/write speed. Which isn't enough of an argument.

joeyguerra commented 8 months ago

I'm fine with keeping this issue open just in case others want to chime in. I'm interested in redesigning the Brain to accommodate multiple Hubot instances. And I'd also like to redesign the Hubot-redis design to not use JSON, but leverage some of Redis' data structures instead.

johnseekins-pathccm commented 8 months ago

I'd love to see Hubot actually able to run more than one instance. If it's just the internal API for the brain causing the issues, that seems solvable. I'm no javascript guy, but if I can help, I'm happy to try.