Implement method for easy chunking of users/members

[Andre601]

When you not have access to the PRESENCE intent (just like me) then you'll face minor issues in combination with caching/chunking of Members. Some library, like JDA for example, mainly use the PRESENCE option to chunk members based on their online status (afaik).

While you can retrieve members on alternative ways (Requesting the info through a REST call/action) does it have disadvantages like longer loading times and - in my case - hitting the rate limit quite fast when the bot is starting.

It would be really beneficial to have a way of easy retrieval of User/Member information to cache/chunk as the only alternative would, in most cases, be to disable said caching/chunking and retrieve the members/users the normal way. This comes with disadvantages like that the Member appears as "unknown" (null) because it isn't cached.

While retrieving it manually would be an option would it cause additional REST calls/actions and also consume more time (And in case of libraries like JDA can also not be 100% done everywhere without the sacrifice of performance or similar).

I hope you get what my point of view here is.

[night]

We intentionally made it harder to obtain full member list data, as at scale it is has become impractical to sync massive lists of members and there are privacy concerns with sharing full lists of guild members. You can still obtain members via chunking via REQUEST_GUILD_MEMBERS if you have guild members intent.

[MinnDevelopment]

In the current state of the API you have to chunk every single guild if you don't have the GUILD_PRESENCES intent set because it doesn't send the entire member list, regardless of the size of the guild. Even if the guild only has 10 members you have to chunk them which becomes a painful process when you are limited to a 120/60 rate limit. As far as I know the vast majority of guilds have less than 100 members, yet the same limits apply.

Privacy concerns make sense but some bots do require the members to be present in cache, for example to track changes of nicknames or roles. The API is built in a way that only tells you that an entity is updated, but doesn't tell you what has been updated. Privacy should be a goal but if the data is necessary and needed then it should be accessible in a reasonable way.

Currently, chunking is essentially impossible due to the extreme rate limits that are applied and the fact that you need to chunk every guild without the presences intent. It cannot be the solution to enable the presences intent (which sends tons of events that aren't relevant) just to be able to properly load members. This certainly doesn't seem like a privacy-oriented system at least.

[Andre601]

I can agree to what MinnDevelopment said.

My bot is in almost 14k Discord server now and chunking Members without access to the GUILD_PRESENCES intent is a near-impossible thing to do as the bot more than once hits the rate limit, causing delays and artificially increase the loading time of what was more or less 40+ seconds to well over 1 minute if not even 2 or more.

I can understand your concerns with privacy, but the near fact that almost 90% of all bots (Rough estimate but I'm quite confident that it's even more) use member information for stuff like User info, welcoming members, logging systems, etc. makes the required stuff mostly a must-have for every bot.

I already pointed out that retrieving the Member information manually through a separate request would be an option, but A) can it not always be used (Or not without overcomplicating stuff for it) and B) does it again artificially increase delays and amount of requests, because yet another request has to be made towards the API.

I myself do have access to the Member intent, but not the Guild Presence one, which causes me major troubles for different features, because the Members can't be chunked/cached on startup.

[advaith1]

I'm pretty sure Discord does not want you to fetch all users for those use cases.

• for user info, you should just fetch that specific member if they arent cached from a previous event
• for welcoming users, fetching users on start wouldn't help at all, and the member info is sent in the event
• for anything triggered by a message, the member object is sent in the event

[MinnDevelopment]

for user info, you should just fetch that specific member if they arent cached from a previous event

I would like to quote myself:

[...] some bots do require the members to be present in cache, for example to track changes of nicknames or roles. The API is built in a way that only tells you that an entity is updated, but doesn't tell you what has been updated.

On another note:

for anything triggered by a message, the member object is sent in the event

I don't understand why you bring this up, nobody said anything about messages. This is completely irrelevant to the issue.

for welcoming users, fetching users on start wouldn't help at all, and the member info is sent in the event

Yeah, this one I can agree with. Welcome messages should work fine with just the GUILD_MEMBERS intent.

[mr-tech]

I personally just find it odd that the members aren't sent in GUILD_CREATE with the GUILD_MEMBERS intent. It's unintuitive to say the least, but I understand that I don't know the infrastructural / business logic reasons for it. If the bot is verified for the GUILD_MEMBERS intent, one would think that its use-case has been approved for knowing of (all) members in a guild and hence a candidate for the field in the GUILD_CREATE event.

Edit: The distinction, to me personally, would be that GUILD_MEMBERS is knowledge of who is in the guild and GUILD_PRESENCES is further knowledge of their usage information / patterns. Someone being able to fetch the whole list with the GUILD_MEMBERS intent is the biggest lender to this.

That said, we can't overlook night's

at scale it is has become impractical to sync massive lists of members

which we do not have immediate control over. For x16 and x64 sharding in particular, ripping those guild member lists in a timely fashion must be really painful. Spreading that out over time (and also forcing people to be more mindful) I can see as being a benefit to Discord's infra.

[muddyfish]

There are also cases such as the one I made a feature request for earlier, #1612, which would really help reduce the requirement for keeping entire caches loaded for some bots.