Add Redis hive - Githubissues

muesli / beehive

A flexible event/agent & automation system with lots of bees 🐝

GNU Affero General Public License v3.0

6.3k stars 324 forks source link

Add Redis hive #272

Closed rubiojr closed 4 years ago

rubiojr commented 5 years ago

Key/value storage and pubsub support using a Redis server.

muesli commented 4 years ago

We could hook this up to Redis pub/sub system. That would actually be incredibly powerful and awesome...

rubiojr commented 4 years ago

Agreed :smile:.

Added very basic subscribe support in 25c3e4b3f0d30e7db3fa30d789b4271bbbd017a9 to deliver events received from a Redis pubsub channel and it opens the door to very interesting possibilities.

Still a WiP, need to make all this a bit more resilient and also thinking about adding publish support before merging this also, so other bees can send bits to the Redis channel.

muesli commented 4 years ago

@rubiojr Awesome work! Just let me know whenever you consider this ready to be merged.

rubiojr commented 4 years ago

I'm almost there. Some minor changes and doc updates and should be good for :eyes:

rubiojr commented 4 years ago

This is ready for :eyes:

To real world test this, I'm currently writing a small tool that:

Posts a URL to Beehive's http server hive
Stores the URL in Redis
Publishes the URL to a redis-channel
Consumes the URL from Redis and downloads the video with youtobe-dl if it's a YouTube URL
Notifies via pushover that a URL was received and a video will be downloaded if the URL is from YouTube
Publishes a message to a Redis channel when the download has finished so Beehive sends a pushover notification

rubiojr commented 4 years ago

Added sample script and Beehive config to https://github.com/rubiojr/beehive-youtube-dl

rubiojr commented 4 years ago

go-redis built-in retries/backoff works quite well so switched to that instead of using an external backoff library, which simplified the code a bit.

rubiojr commented 4 years ago

Just realized that my last force push to incorporate master changes didn't work as intended, reverting some of the changes I had pushed... fixing in a bit.

rubiojr commented 4 years ago

Just realized that my last force push to incorporate master changes didn't work as intended... fixing in a bit.

Fixed in https://github.com/muesli/beehive/pull/272/commits/351b85af354892d24e14bd88310e9eacdab9fe0d

rubiojr commented 4 years ago

Thanks y'all for the :eyes: !