search

jaraco / irc

Full-featured Python IRC library for Python.
MIT License
390 stars 84 forks source link

Question: High volume bot usage #164

Closed utoxin closed 4 years ago

utoxin commented 4 years ago

I'm trying to pick a new framework to replace an old and extremely annoying framework that I'm using for a bot right now, and I'm trying to figure out if this will work for me.

My main question, is how well this framework will deal with heavy traffic. My bot runs in 100+ channels, often with people talking in all of them, and it does a lot of interaction with them. Will the AsyncIO mode handle that well? It sounds promising, but I'd rather not get deeply into a rebuild before finding out I was wrong. :)

MattBroach commented 4 years ago

So, I've never done any load testing on the asyncio-based version, so I can give any raw numbers. However, I've been running it in production without any load issues, although at most we're dealing with a couple dozen channels at a time rather than 100+. Perhaps @jaraco has done some load testing on the old socket-based version, to at least give some context?

No matter how good the underlying IRC library is, though, eventually if your traffic is high enough you'll have to put it behind a load balancer and run multiple instances of the software. Specifically where that cutoff number is with the asyncio version of this library I unfortunately couldn't say right now. If anyone DOES do some load testing our have some representative numbers from production, I'd certainly be interested in seeing them.

jaraco commented 4 years ago

At my old firm, we ran pmxbot, built on this library against Slack with hundreds of channels and many hundreds of users and other bots, and it performed well (socket/select model) with just one process. Where we ran into performance bottlenecks was with other services (sqlite database).

I don't have any other data than that.

utoxin commented 4 years ago

Cool. Thanks for the feedback. That sounds like it should serve my needs then. Now I just need to buckle down and start the rewrite. This Java framework is nice... but I'm sick to death of fighting with getting it to build on new systems.