search

42wim / matterircd

Connect to your mattermost or slack using your IRC-client of choice.
MIT License
294 stars 60 forks source link

Reduce calls to Mattermost server by caching replies #513

Closed hloeung closed 1 year ago

hloeung commented 1 year ago

Replaces https://github.com/42wim/matterircd/pull/508. For each message that's a reply to a thread, we make an extra API call to get the parent message to use as reply text. This caches that to reduce requests to the MM server.

hloeung commented 1 year ago

Memory usage is pretty good and remains quite constant. This is with ShortenRepliesTo = 20 but shows that there isn't any memory leaks:

$ while echo -n "$(date): "; do ps aux | grep './m\atterircd'; sleep 1800; done
Tue Nov 22 10:29:56 PM UTC 2022: ubuntu   52779  0.1  0.4 724940 32732 pts/2    Sl+  02:46   2:18 ./matterircd
Tue Nov 22 10:59:56 PM UTC 2022: ubuntu   52779  0.1  0.4 724940 32608 pts/2    Sl+  02:46   2:21 ./matterircd
Tue Nov 22 11:29:56 PM UTC 2022: ubuntu   52779  0.1  0.4 724940 32608 pts/2    Sl+  02:46   2:24 ./matterircd
Tue Nov 22 11:59:56 PM UTC 2022: ubuntu   52779  0.1  0.4 724940 32608 pts/2    Sl+  02:46   2:28 ./matterircd
Wed Nov 23 12:29:56 AM UTC 2022: ubuntu   52779  0.1  0.4 724940 32680 pts/2    Sl+  Nov22   2:31 ./matterircd
Wed Nov 23 12:59:56 AM UTC 2022: ubuntu   52779  0.1  0.4 724940 32764 pts/2    Sl+  Nov22   2:34 ./matterircd
Wed Nov 23 01:29:56 AM UTC 2022: ubuntu   52779  0.1  0.4 724940 32764 pts/2    Sl+  Nov22   2:37 ./matterircd
Wed Nov 23 01:59:56 AM UTC 2022: ubuntu   52779  0.1  0.4 724940 32764 pts/2    Sl+  Nov22   2:40 ./matterircd
Wed Nov 23 02:29:56 AM UTC 2022: ubuntu   52779  0.1  0.4 724940 32764 pts/2    Sl+  Nov22   2:43 ./matterircd
Wed Nov 23 02:59:56 AM UTC 2022: ubuntu   52779  0.1  0.4 724940 32764 pts/2    Sl+  Nov22   2:47 ./matterircd
Wed Nov 23 03:29:56 AM UTC 2022: ubuntu   52779  0.1  0.4 724940 32720 pts/2    Sl+  Nov22   2:50 ./matterircd
Wed Nov 23 03:59:56 AM UTC 2022: ubuntu   52779  0.1  0.4 724940 32720 pts/2    Sl+  Nov22   2:53 ./matterircd
Wed Nov 23 04:29:56 AM UTC 2022: ubuntu   52779  0.1  0.4 724940 32752 pts/2    Sl+  Nov22   2:56 ./matterircd
hloeung commented 1 year ago

This removes all the complexity of maps/counters/locking and lets the 3rd party library deal with it. The keys should be unique enough to just put them in the main cache instead of mapping it per channel.

WDYT ?

Good call/advice, I've updated the PR switching to using the 3rd party HashiCorp LRU library.

I think the advantage of using per-channel caching is that a channel that sees less activity would not have cached entries pushed out of the cache because of having channels with more activity. But yeah, this simplifies the code (less code, less bugs and all that), we can further tune the cache size if needed.

hloeung commented 1 year ago

Thanks, I'm running with it now but not in a rush to have it landed so happy to wait for additional testing :+1: