search

HarukaNetwork / HarukaX

Telegram bot written in Go, built to scale.
Other
31 stars 31 forks source link

Cache admins, Instead of API calling. #4

Closed soekarnohatta closed 4 years ago

soekarnohatta commented 4 years ago

it is inefficient to call API everytime. We can use some caching methods to cache the admins instead of calling API. Redis or BigCache may help us to cache some bit of data like admins and manymore

RealAkito commented 4 years ago

@soekarnohatta Agree, That's how Rose did it as well.

baalajimaestro commented 4 years ago

On the API end, its not too heavy to fetch admins, its only heavy to ban and unban. I dont see a point caching this. The cache can anytime go outta sync

RealAkito commented 4 years ago

Though It would speed up

soekarnohatta commented 4 years ago

On the API end, its not too heavy to fetch admins, its only heavy to ban and unban. I dont see a point caching this. The cache can anytime go outta sync

There must be a gap to get list of admins when using API calls, cache may speed up, especially when the bot has a lot of users...

RealAkito commented 4 years ago

On the API end, its not too heavy to fetch admins, its only heavy to ban and unban. I dont see a point caching this. The cache can anytime go outta sync

Imagine having bot like Rose with 4 millions users..

baalajimaestro commented 4 years ago

On the API end, its not too heavy to fetch admins, its only heavy to ban and unban. I dont see a point caching this. The cache can anytime go outta sync

Imagine having bot like Rose with 4 millions users..

Only on that case, you need a cache, gingko hardly i even used lmao --edit-- Even Rose's stuff doesn't make sense to me tbh. They can cache other stuff stored on main db to redis rather than admins. Its seriously not at all heavy. You wont get rate limited for fetching the users even 6969 times

RealAkito commented 4 years ago

@baalajimaestro Well, Who knows what would happen? Do it sooner than later.

gotenksIN commented 4 years ago

@RealAkito this ^

SphericalKat commented 4 years ago

I agree caching is necessary in the long run for scalability, however I'm pretty swamped rn, and the welcome module implementation is still WIP.

I'll be very glad to merge anyone PRing cache-related features.

gotenksIN commented 4 years ago

@baalajimaestro @Oc69bot is very sad :^)

RealAkito commented 4 years ago

Alright then

baalajimaestro commented 4 years ago

@RealAkito edited previous comment

jumatberkah commented 4 years ago

@baalajimaestro Imagine you ask something to a teacher many times, not only the teacher get tired, but you also get tired because of asking many times.

This cause an unnecessary API calls and a waste of time,bandwidth...

baalajimaestro commented 4 years ago

@baalajimaestro Imagine you ask something to a teacher many times, not only the teacher get tired, but you also get tired because of asking many times.

This cause an unnecessary API calls and a waste of time,bandwidth...

Its not heavy call at all.

RealAkito commented 4 years ago

@baalajimaestro Imagine you ask something to a teacher many times, not only the teacher get tired, but you also get tired because of asking many times. This cause an unnecessary API calls and a waste of time,bandwidth...

Its not heavy call at all.

In Rose, The API gonna delay if the bot is sending too much requests. It indeed the heavy and unnecessarily call to API at all, caching it would completely prevented the problem

SphericalKat commented 4 years ago

I'm going to lock this conversation with these closing comments:

  1. Caching is a very real necessity
  2. Ginko does not maintain a list of admins, rather, it checks the ChatMember object.
SphericalKat commented 4 years ago

Now, I'll be accepting PRs for caching functionality; however, the caching should be applied somewhere it's actually necessary.

Just for reference, the heaviest calls that Ginko faces right now is the database. We have practically no network requests other than the ping function.