Charcoal-SE / SmokeDetector

Headless chatbot that detects spam and posts links to it in chatrooms for quick deletion.
https://metasmoke.erwaysoftware.com
Apache License 2.0
474 stars 182 forks source link

Shut off Bioinformatics #1665

Closed Undo1 closed 6 years ago

Undo1 commented 6 years ago

Per https://chat.stackexchange.com/transcript/message/43151745#43151745, we're shutting off https://bioinformatics.stackexchange.com/. Need a decent way to do that - probably filtering them out at ws.py, but open to suggestions.

AWegnerGitHub commented 6 years ago

For the record, I am very much opposed to this. This entire situation is due to how a request for disclosure was posted. The Smokey detection was correct.

This sets a bad precedent. We are no longer a network-wide solution to spam on SE. Doing this means that individual communities can come to us to have us ignore detections. This isn't a request for no-autoflags. This is a request to turn Smokey completely off on a site. No posts to chat even.

Discussions can no longer occur at the MSE level for large system changes. It must be done at the individual meta level because changes can and will only impact certain sites.

Undo1 commented 6 years ago

Here's the question we need to answer - three options:

  1. Enmity with a few moderators and the tension that brings.
  2. Allow moderators to demand we tune technical bits of our project to their liking, without numbers to support it, or
  3. Drop the whole site and force a meta post about it on that site.

None of these are awesome, and it's unfortunate that we have to pick one - but I don't see any other options.

I definitely don't intend this to be eternal - but I'd rather drop one tiny site forever than set a 'moderators can meddle with reasons' precedent.

Maybe I'm missing something - thoughts?

AWegnerGitHub commented 6 years ago

Option 1 is already occurring. It's a problem that needs to be looked at and resolved, but not in this issue. Option 2 I'm not a fan of at all. It's also already occurring with both this issue and the autoflag discussion that is on going. Option 3 I could support, if we put a time limit on both how long a site isn't monitored and how long until the meta post is made.

I'd add a stipulation to Option 3 though, that's going to make this change more complicated: Smokey should still monitor that site, but not report it to the chatrooms or the Metasmoke review queue. It should still be reported to metasmoke though. We need to know how many posts are not being reported and possibly how long it takes to remove those posts. It's information that we can use to support our discussions on whether or not to support this in the future.

Undo1 commented 6 years ago

Option 3 is pretty much what we have here - terdon said he'd throw a meta post out. If he doesn't deliver a meta post, we'll go from there. I don't know of any way to calm the waters with moderators having concerns like this other than having per-site meta posts. I don't think they'll accept a network meta post as being authoritative anyway.

And remember, we're dealing with 150+ sites here. The vast majority aren't problematic. It's the vocal minority we hear.

quartata commented 6 years ago

We've turned it off for BI but I'm not sure this should be closed yet

terdon commented 6 years ago

Since I was the moderator who started this, I feel I should clarify a couple of things here. First of all, I did not ask for and do not want to have the entire system turned off for BI! All I wanted was for one small facet of it to be turned off: the reporting of "self promotion" posts in the Charcoal HQ chat room. I have now opened a meta post about this and its response, such as it is, has been as positive as I expected. However, given how small and new the site is, meta discussions are unfortunately rarely very active though, so it's hard to draw conclusions.

What I wanted was to avoid having users who aren't familiar with the site coming in and leaving coments explaining "the rules" before we, the BI community, have had a chance to decide what our rules will be. Flagging spam is one thing and a thing that you all have repeatedly proven that you do brilliantly. The false positive rate on spam flags is incredibly low and smokey is one of the best things about the SE network.

But each site has its own memes, customs, unwritten rules and character. This is especially true of the smaller sites where most users know each other and are still developing a sense of community. On such sites, the arrival of what can be perceived as "outsiders laying down the law" can be a problem. All I wanted was for self promotion posts—specifically—not to be reported in the charcoal chat room so that the users actually involved with the site could be given the chance to deal with such post and decide how such posts should be dealt with.

I think you really should consider that while spam and R/A posts are usually clear cut, other forms of possibly problematic content are less so. Also, while it is easy to demonstrate the (enormous) positive effect that autoflags have had on the quality of posts across the network, assessing the effect of comments is almost impossible. Comments have a psychological effect since they are public and read by all. This effect is intangible and impossible to quantify.

More generally, it might be a good idea to consider making your system more modular and allowing for different rules on different sites. While some things, like spam and R/A, follow the same rules across the network, other things do not. It seems reasonable to be able to tweak certain aspects of what you do in a site-specific manner.

Finally, I also wanted to point out that this idea that there's "enmity" against charcoal from mods is completely wrong. At least as far as I'm concerned. Yes, some mods, myself included, have raised some concerns, but interpreting "hey, maybe this isn't a good idea" as enmity seems far fetched. If that's how you feel, then we obviously haven't been expressing ourselves clearly. As a moderator on three sites, one of which is a very popular spam target, I am very well aware of how good smokey is and I appreciate your efforts enormously. I remember all too well how it was to be under spam attack on Ask Ubuntu, for example, and spam is basically a solved problem now because of the great work you folks do. I would absolutely hate it if smokey were to be discontinued and I am pretty sure that so would all other mods.

Undo1 commented 6 years ago

@terdon I should probably also clarify one thing - why I didn't want to shut that reason off, and went with shutting the whole site off until we have clarity. I had a hard time articulating it before, now it's clearer:

We are able to do as well as we can because we don't have obligations to anyone around the lower-level technical stuff like reasons. I'm concerned that if we were to start making promises to communities about reasons, etc., we're going to build up a web of dependencies. That would slow us down, make us lose agility. Agility is the only advantage we have over an SE-baked system.

There are all kinds of concerns that come with a seemingly simple "Shut this reason off for us, please": It's easy to do now, certainly. But once we've made that promise, we have to remember it and debate it in the future. For example, suppose we find another way to detect undisclosed self promotion. Would BI want that? Maybe, maybe not - now we have to debate it, find a moderator to ask, or post something on meta. And that's assuming we remember that a month, a year, two years from now.

Essentially, I don't know that we can make a commitment like that in good faith, knowing that we probably can't reliably sustain it at our pace.

Maybe I misinterpreted what you wanted as more authoritative than you meant it to be. I could have said "sure, but I can't guarantee it won't be back in some form next month or next year". Maybe I should have, and read it wrong.

It seems the real issue here is comments, though. I think we're fine with just not leaving comments on sites we're not familiar with. There was discussion towards that point in the last few days.

terdon commented 6 years ago

@Undo1 yep, absolutely. The real problem was comments. You folks are brilliant at flagging, no argument there.

ArtOfCode- commented 6 years ago

So... this feels like a problem that can be solved socially, without resorting to disabling Smokey for BI entirely. Seems clear to me that the actual problem here is comments, specifically comments on sites that we don't fully understand (which is most sites; there are always a few Charcoal members familiar with each site, but as an organisation we're not).

Instead of shutting BI off, does it work to just put a blanket "don't comment unless you're familiar with the site" policy in place?

angussidney commented 6 years ago

Generally, when I post comments on borderline posts like this, it's not really to help the user, but rather to show the moderator that's handling my flag the reason why I'm flagging it.

I'm happy to follow Art's suggested policy, but I'd like to know from the moderator's perspective: would you prefer me to only flag as spam, and provide no context, or use a modflag to indicate my thoughts?

terdon commented 6 years ago

@angussidney certainly flag instead of commenting please, yes. The flag interface doesn't show us comments, so leaving a comment in order to communicate with a moderator won't work. Sites can get several hundred flags a day, so opening each post to check it in detail (which is the only way we might happen to see the comment) isn't usually feasible.

As for spam, spam flags are better when it is clearly spam. For things that are more borderline, please use a mod flag and explain why you are flagging so the mod who handles it can understand.


@ArtOfCode- "don't comment unless you're familiar with the site" sounds brilliant, yes. Thanks!