python-discord / bot

The community bot for the Python Discord community
https://pythondiscord.com
MIT License
1.35k stars 667 forks source link

"Junior Developer" feature for new members of our community #1258

Open SebastiaanZ opened 3 years ago

SebastiaanZ commented 3 years ago

One of the issues we're facing in terms of moderating the community is that we see people entering the community just to troll or be obnoxious in other ways. This includes dumping NSFW imagery in high activity channels like #python-general and posting spam links.

Junior Developers

A suggestion that has been made a couple of times, both by regular community members as well as staff members, is restricting permissions for new members of the community. Basically, new members (defined in terms of membership duration and/or server activity) will not have permissions to perform certain actions, such as posting images, embedding graphics using links, or posting attachments in parts of the community.

One way to do that is by assigning new members a special role (e.g., "Junior Developer") that will automatically be removed when certain conditions are met. This mirrors the current the design of Voice Verification, where members can explicitly obtain a role when they meet certain criteria.

Implementation / Discussion

Before we can implement such a feature, we'll need to work out a couple of details and decide whether or not we want to have such a system in the first place. The latter consideration also depends on the estimated implementation and maintenance costs of the preferred design, compared with the gains it will give in terms of community moderation.

Who are junior developers?

One of the questions we need to answer is when someone should lose their "new member" status. Experiences from our voice gating system tell us that the message count is a more effective measure than the raw "membership duration" value. Basically, it appears as if trolls may join servers quite some time before starting to troll, but will rarely have accrued a substantial message count. A downside is that we've also seen that a message count requirement leads to members spamming messages to up their message count.

In short: We need a smart solution that does not incentivize other types of bad behavior.

Implicit or explicit leveling up

Our voice verification system works with explicit gating: Not only do members have to meet certain criteria, they also have to explicitly opt-in to our voice channels. We decided that, for voice verification, this explicit step is important: People need to notice that we have a special voice channel policy and that speaking permissions are a privilege that we could take away as well.

Personally, I think the "new member" -> "regular member" transition should be implicit: Instead of going through another verification system, members simply level up once they meet the requirements we define. I think explicit gating for a feature that will affect so many users is not only a bad user experience, but will also come with a lot of constant moderation/admin costs in helping users who require assistance with the system. (We already see that with the voice gating.)

Other considerations

Please do add considerations I've missed.

Design

I don't have a lot of design suggestions yet, as the design depends on the considerations above. I do want to mention that Metricity allows us to have activity-based criteria, but we'll probably have to think about doing that in a smart way to not overload the system with requests.


Please note: This feature is not ready to be implemented; we first need to come up with a proper design and then decide if such a system is worth the implementation and maintenance costs. Please add your thoughts to this issue.

ks129 commented 3 years ago

I think this should have like 1 month and ~50 messages requirement. There should be also way to mark users dangerous, what means that Junior Developer role will be not removed from this user until staff mark user again safe.

Jayy001 commented 3 years ago

This sounds like a great idea, I have a few suggestions.

Not sure if this has been said before but maybe like a rep system? If somebody's helps you then you give them a rep, if the reach a certain ammount they get the role.

Another suggestion (which is quite basic) is a 1 week to get the role, you could also make it so you need a certain ammount of msgs in a help channel.

This last one might be difficult, but some sort of machine learning program to work out who's actually talking about programming and helping other users and the other people who just shitpost etc. It could gather all the messages and then see what channels they have been posted in, if they are in code format or if they have certain keywords in them etc.

Overall, as I said it sounds like a great idea to be implemented! 👍

Brocoliman commented 3 years ago

This sounds quite a good idea. I have a few ideas that could mimic MEE6. I personally think the "leveling" system can work well. However, as you addressed people could spam.

So to clarify, these are the general ideas for the basic plan of the "level" system

SebastiaanZ commented 3 years ago

In board terms, I think we're looking for a simple system that just adds a bit of a barrier for trolls, not an extensive reputation/leveling system. This means that we want to have simple criteria that we use to determine that someone should have those permissions.

I'm currently leaning towards a system that combines:

We could even scale the message count by message content length to favor longer messages, but I'm not sure if that's even necessary.

Brocoliman commented 3 years ago

Yes, we can make it so the bot checks all of the user's sent and undeleted messages, (excluding the non-conversation channels,) when the "command" is used. I'm pretty sure you could do that without keeping track of the "levels" in a database/file. (Statbot is able to do that, you could probably allow the bot to do that with Intents or smth like that)

Den4200 commented 3 years ago

Yes, we can make it so the bot checks all of the user's sent and undeleted messages, (excluding the non-conversation channels,) when the "command" is used. I'm pretty sure you could do that without keeping track of the "levels" in a database/file. (Statbot is able to do that, you could probably allow the bot to do that with Intents or smth like that)

@Brocoliman There shouldn't be any need for another bot, we have Metricity, which keeps track of all those statistics (and more) already.

A similar sort of system to the one proposed here is our voice gate. I think something like this, but without a command to pass the gate, would be a pretty good solution. Users should automatically pass the gate and be given the appropriate permissions once they reach the set requirements.

Brocoliman commented 3 years ago

Nope, I don't mean another bot. It can just be a new feature on one of the existing bots. I will rephrase my original comment, since it might have been unclear for some of you

Brocoliman commented 3 years ago

This sounds quite a good idea. I have a few ideas that could mimic MEE6. I personally think the "leveling" system can work well. However, as you addressed people could spam.

So to clarify, these are the general ideas for the basic plan of the "level" system

  • People gain "experience" if they talk
  • To counter spam, make it have a cooldown like MEE6 (e.g. it only adds experience points every 1 minute you make a message)
  • To make what messages the bot counts as actual conversation, you can limit the "experience channels" to conversation channels like #python-general, topic channels, and such.
  • To improvise on the suggestion above, you may have to make an auto-mod for spamming onto the Python Bot(If it already has one, I have more suggestions to it). This will help. For example, even if someone spams in a lesser-modded channel, the python bot will flag the spamming and then, remove an amount of experience from the user who spammed. This amount could be how much spam the user sent.
  • To be honest, I don't think you should have to be voice verified to post images, because some people don't, and don't want to, use the voice channels. But, you can also do a similar verification system. Once a user gets to a certain level, they are allowed to use the verify channel for turning from a "Junior Developer" to a "Developer".

Same ideas apply, except to not use a "level and experience" system. Instead, basic idea is like this. When a user triggers the verification command, the bot will check how many messages the user sent in the conversation channels, either since they joined, or in the past. If the user has been automodded or muted for things like spam or by moderators in the past month, then they will not be eligible for verification.

Feel free to change any of the "past month" or such.

gregoriB commented 3 years ago

I think "new member" would be a better role name. It's pretty straightforward for what it is and doesn't state their level of development experience.

Akarys42 commented 3 years ago

I think "new member" would be a better role name. It's pretty straightforward for what it is and doesn't state their level of development experience.

The idea would be that it fits with our current @Developer role.

gustavwilliam commented 3 years ago

I think @gregoriB has a point. While the role name with “Developer” in it would fit the Developer role, I still think it could be viewed as a judgment of their skills. I think @New Member would probably be a better name, unless there’s some reasoning I’ve missed.

ChrisGallardo commented 3 years ago

I agree with @gregoriB and @gustavwilliam, I think naming the role "Junior Developer" is talking about skill level, which has nothing to do with the role. I think also some people would wonder and ask why they are junior developer, which won't happen with a more self-explanatory role name like "New Member". I think this is a good idea though 👍🏻.

ChrisGallardo commented 3 years ago

Instead, basic idea is like this. When a user triggers the verification command, the bot will check how many messages the user sent in the conversation channels, either since they joined, or in the past. If the user has been automodded or muted for things like spam or by moderators in the past month, then they will not be eligible for verification.

@Brocoliman I think we are against having a verification command, because as @SebastiaanZ said:

Personally, I think the "new member" -> "regular member" transition should be implicit: Instead of going through another verification system, members simply level up once they meet the requirements we define. I think explicit gating for a feature that will affect so many users is not only a bad user experience, but will also come with a lot of constant moderation/admin costs in helping users who require assistance with the system. (We already see that with the voice gating.)

The change/level up should be automatic and not require the user to do anything other than meet the requirements.

SebastiaanZ commented 3 years ago

I'm completely on board with naming it "new member". The "Junior Developer" was more WIP name because it sounded a bit cooler.

Akarys42 commented 3 years ago

Alright, so we would still have the same idea with the "new member" role being given instead of "Developers", or we can have both? I'm not sure if role overrides would work with both.

gustavwilliam commented 3 years ago

I think they should both be given. Having one United role for all verified members seems optimal, and the "New Member" role would simply add additional restrictions. If it's simply above the "Developers" role in rank, we can change the permissions for the users. It would of course also allow us to modify the bot's behavior if we wanted to, while still having a "verified member" role (Developers).

Akarys42 commented 3 years ago

You can override permissions with a deny? I’m not 100% sure if that’s possible.

gustavwilliam commented 3 years ago

If a lower role has a deny, you can grant the permission with a higher role. If a lower role has a permission granted, you can deny it with a higher role.

Akarys42 commented 3 years ago

Alright nice, it seems like having both is a better option indeed then.

scragly commented 3 years ago

If a lower role has a permission granted, you can deny it with a higher role.

You may want to confirm this for sure, as this is not how I believe this works. I'm pretty sure channel role overrides are additive, not hierarchical. Once you've granted a role-specific override to allow a permission, you can't deny it with another role, even if it's above it in the list.

gustavwilliam commented 3 years ago

@scragly I was referring to general role permissions. I haven't looked into channel specific permissions.

After testing this, it seems like the permissions are indeed additive.

ioistired commented 3 years ago

General role permissions don't even support denies to begin with ?_?

SebastiaanZ commented 3 years ago

I think they should both be given. Having one United role for all verified members seems optimal, and the "New Member" role would simply add additional restrictions. If it's simply above the "Developers" role in rank, we can change the permissions for the users. It would of course also allow us to modify the bot's behavior if we wanted to, while still having a "verified member" role (Developers).

The role hierarchy should be irrelevant for what we want to achieve.

If a lower role has a deny, you can grant the permission with a higher role. If a lower role has a permission granted, you can deny it with a higher role. I was referring to general role permissions. I haven't looked into channel specific permissions.

There's no "deny" in the server-wide role permissions, only a "non-granted" or "granted" toggle for each permission. This means that as soon as one of your roles has a "grant" for a permission, you've got that permission. There's no way to then "undo" that grant with a role higher in the hierarchy using the server-wide permissions. This is why the "Muted"-role does its work using permissions overwrites put on channels.

When it comes to overwrites, simplifying it by talking only about role-based overwrites, you can take away a global permission with a channel-specific overwrite (including a CategoryChannel overwrite). However, if one of your roles gives you an explicit "allow" channel overwrite for a permission, you cannot take that "allow" away with a "deny" channel overwrite, regardless of hierarchy.

This is why the Muted-role originally did not disallow reactions in some channels, as we granted those reaction perms using an overwrite for the Developers-role (despite the muted role being higher in hierarchy). We now allow reactions for developers on a server-wide basis, which does allow us to take it away with an overwrite for the Muted role.

gustavwilliam commented 3 years ago

Sorry about that and thanks for the clarification. I really thought that was how it worked. It would be quite convenient if Discord actually added an explicit deny feature, though that's not really a topic for this issue.

15696 commented 3 years ago

Do we want a command to reverse a "level up"?

I think that an infraction such as a mute or warn should reverse the progress slightly, so that it should be harder for someone who is trolling and breaking a lot of rules to lose the "New Developer" role as they can not be trusted proved by the amount of infractions they have had.

15696 commented 3 years ago

Not sure if this has been said before but maybe like a rep system? If somebody's helps you then you give them a rep, if the reach a certain ammount they get the role.

I agree with this also, something like if and when a user has received help from the new developer, the person that received help could then choose to give that user XP because again, it proves that they can be trusted with extra permissions.

I do not know how this will work though if a user creates or uses alternative accounts to increase their reputation, so maybe you will need to have the Developer role in order to give someone reputation, or they could click on a link that the Python bot would DM them when they join to make sure the IP addresses of the two accounts don't match? This complicates the idea greatly, though.

And the idea of clicking on an IP logger will not appeal to a lot of people, so I think that part would not work.

SebastiaanZ commented 3 years ago

I think it's important to appreciate we're not talking about a true "leveling system" like you see in many other bots, with reputation and very visible "level" embeds. In fact, the core of the system is about having two tiers:

As such, it's mostly a preventative measure in the area of moderation rather than having a member-facing level system to boost the apparent status of a member. So, it's not about XP, it's not about reputation, it's not about having very visual signs of people leveling up, or having a certain status. It's purely about limiting the permissions of people when they've "just" joined the community to prevent abuse.

Akarys42 commented 3 years ago

What permissions would we include in this new membership level though? Originally it was designed to re-allow images in pygen, but I think it would be fair to say that it is a fast enough scrolling channel without images, and with those, it would be even worst.

SebastiaanZ commented 3 years ago

I don't have a clear answer to that, but I was mainly reading the previous responses and wanted to make sure that this issue is not about having various reputation/status levels, if we even decide to do it.

I'm not sure we need such a system at the moment, but there may be areas to consider:

And so on.

It's just that the initial motivation, selective gating of only py-gen, has gone away.

mathstrains21 commented 3 years ago

Something to be though about here as well potentially is: do we apply these constraints to current members who haven't completed these requirements? maybe a period where new joins gets this applied, and then at the end of that, we do a mass apply to server members?

also, if a member loses the requirements by deleting messages or getting infracted, should they get given the new members role again?