Team-TAU / tau

TAU- Twitch API Unifier, a containerized relay/proxy to unify the WebHook- and WebSocket-based real-time Twitch APIs under a single (local) WebSocket connection.
MIT License
154 stars 39 forks source link

feat: support ext redis like redislabs.com #36

Closed mtfoley closed 3 years ago

mtfoley commented 3 years ago

Per conversation with @FiniteSingularity and @0-vortex on discord, I tried out Tau with redislabs.com free tier. These changes to common.py and the .env file got the job done, but tau-redis docker container is still spinning up. Is there such a thing as an "optional" depends_on in Docker?

RE: free tier, it's rated for 30MB, and you start off with almost 2MB. Spinning up and down is no big deal, so like we talked about, we'd just have to kind of test it on a stream to see how it stacks up against the free tier.

RE: lag, it spun up the status checkmarks within a few seconds and I didn't notice any appreciable lag when i used the test buttons.

0-vortex commented 3 years ago

REDIS_ENDPOINT removes the necessity for REDIS_SERVER=localhost/tau-redis - local docker compose (default) should remove the redis depends_on and be used to spring up missing binaries; for compatibility purposes the current docker compose file should be duplicated into compose.redis-local and compose.all-services IMO leveraging builds wich arg and env conditionals only

mtfoley commented 3 years ago

Does it make sense to leave REDIS_SERVER in place for backwards compatibility for current users?

0-vortex commented 3 years ago

if .env is not validated by some ruleset no, it would only cause confusion for new users going forward; we could even check in the connection middleware if our redis is redislabs and do some magic there and remove all user complexity :D

FiniteSingularity commented 3 years ago

Made some progress on this last night. I'm not entirely sure why, but calls to redis from the worker process seem to have significant lag, while calls from the daphne served code work without issue. Thus, I will be making the following code changes (which should be done anyway):

  1. Add REST endpoints for updating twitch webhook and websocket connection status values.
  2. Remove all database write/update functions from worker process. Rather, when status values are updated, the worker process should push an updated value to the proper endpoint from 1. This has 2 advantages: first using a remote Redis server should work, second, now the server process is the only process which will change database entries.
mtfoley commented 3 years ago

@FiniteSingularity I'm glad you made some progress. I'm sorry I missed the early parts of the stream, and I was watching the playback today. I saw that at one point you had some of the channels codebase open and I thought to myself "he is either going to rewrite something significant, or file a bug with channels, because here there be monsters. :-) "

I see you've got a feature branch going for this, so If you'd like me to make any changes to this PR, I'm game.

mtfoley commented 3 years ago

@FiniteSingularity I think I managed to stumble thru the merge conflicts on that login page. Hope this works out well for you!