42wim / matterircd

Connect to your mattermost or slack using your IRC-client of choice.
MIT License
294 stars 60 forks source link

Fix fetching list of channels #491

Closed hloeung closed 1 year ago

hloeung commented 1 year ago

In addition to https://github.com/42wim/matterircd/pull/490, this fixes the retrieval of the list of the list of channels.

Per https://github.com/42wim/matterircd/issues/243 & https://github.com/42wim/matterircd/issues/248, the Mattermost server has a hardcoded maximum page limit of 200. So when adding debugging output, I get:

[2022-11-05T13:54:39+11:00]  INFO matterclient: found 158 channels for user in team canonical
[2022-11-05T13:54:39+11:00]  INFO matterclient: found 200 public channels in team canonical

With this, we get higher numbers:

[2022-11-05T14:14:50+11:00]  INFO matterclient: found 158 channels for user in team canonical
[2022-11-05T14:14:55+11:00]  INFO matterclient: found 1116 public channels in team canonical

This is also what's causing https://github.com/42wim/matterircd/issues/479

hloeung commented 1 year ago

There are other places hardcoding page 0 with a larger page limit (5k and 50k) but that can be addressed in future PRs.

hloeung commented 1 year ago

This is mostly copied from the fix for users in https://github.com/42wim/matterircd/issues/244 / https://github.com/42wim/matterircd/commit/8e043ed7e0bdeb7bfad6d6e966cec16b799558ba

42wim commented 1 year ago

@hloeung could you try with https://github.com/matterbridge/matterclient/pull/1 (based upon your code) Works fine for me, but test it before I merge :)

hloeung commented 1 year ago

@42wim, Sure! Trying that now.

hloeung commented 1 year ago

@42wim LGTM. Also cleaner fix, thanks.

One thing, can we make the found X messages INFO?

[2022-11-05T21:04:16Z]  INFO matterclient: Found version 6.6.0.6.6.0.96a259e895e1143a8e843b070c21019a.true
[2022-11-05T21:04:21Z] DEBUG matterclient: found 947 users in team canonical
[2022-11-05T21:04:21Z] DEBUG matterclient: initUser(): found our team canonical (id: sqmc4sz45prypmkfctwynm5yjr)
[2022-11-05T21:04:26Z] DEBUG matterclient: found 158 channels for user in team canonical
[2022-11-05T21:04:26Z] DEBUG matterclient: found 1316 public channels in team canonical

I think those are useful or nice to see.

hloeung commented 1 year ago

Oh, also FWIW, on working on this, I noticed quite a few calls to UpdateChannelsTeam(). With this change, it means even more requests due to pagination. Maybe something to consider for optimising and improving next.

42wim commented 1 year ago

wrt Info vs Debug, it's very subjective of course, but I think it's more debug info than actual needed info. On the fence though :)

hloeung commented 1 year ago

heh no problem. Filed https://github.com/42wim/matterircd/issues/492 for the frequency of these calls

42wim commented 1 year ago

Superseded by #494