mastodon / flodgatt

A blazingly fast drop-in replacement for the Mastodon streaming API server
GNU Affero General Public License v3.0
86 stars 8 forks source link

Share a single Redis connection #14

Closed codesections closed 5 years ago

codesections commented 5 years ago

This PR significantly improves performance while also simplifying the code—my favorite type of PR!

I previously believed that Redis supported only a limited number of Pub/Sub connections on a single TCP connection. However, after doing more research and confirming on the Redis IRC, I learned that this is mistaken—all of our Pub/Sub subscriptions can easily share a single TCP connection.

Learning this allows us to simplify the code structure considerably: instead of needing a Receiver for each Pub/Sub subscription, we can use a single Receiver for all subscriptions. Additionally, since the creation of the TCP connection now happens only once (when the server is launched) we can connect synchronously, which further simplifies the code.

This change significantly improves performance because we no longer create multiple connections, which should mean that handling a large number of concurrent connections no longer causes errors related to having too many TCP connections open.