SOBotics / FireAlarm

A Stack Exchange chatbot to catch low-quality posts.
https://bonfire.sobotics.org
MIT License
14 stars 8 forks source link

Reduce the rate of questions feed into the room by introducing a delay and checking for close votes #1

Open NathanOliver1 opened 7 years ago

NathanOliver1 commented 7 years ago

The rate of question the bot finds is a little high. I am wondering if we could mitigate this by putting a delay between when the question is posted and when the bot notifies the room. If we could have it wait X minutes (I'm thinking like 15-30 minutes) and then if there are no close votes then go ahead and notify. If it has close votes or is already closed then the tag is working on it or has done its job and there is no reason to let us know.

Just like [cv-pls] we should be giving the people in the tag the opportunity to handle the trash before we step in.

double-fault commented 7 years ago

Seems like a good idea, though I would try to keep the tags of the post in use to. I'm talking about cases in which the bot detects a bad q on a low traffic tag, and that gets a close vote only because it was on the front page and someone notices it, but then it rots because it is on a low traffic tag and no one notices it again after it goes off the front page.

To solve this, I think that you should check if the tags in the post match any tag in a list of predefined tags. If it doesn't, we report, otherwise we follow what you said. This IMO will stop q's in low-traffic tags from rotting away.

I already have a list of programming tags in the C version made with the help of Triplee, which might be a good starting point for creating that predefined list.

NathanOliver1 commented 7 years ago

@Fortunate-MAN I like that. The smaller tags could use some more oversight. Not sure what we want to call small but it is a great idea.

double-fault commented 7 years ago

@NathanOliver1 maybe see the total questions per tag and classify them as per that?

NathanOliver1 commented 7 years ago

@Fortunate-MAN We could do that or if its easily query-able go off how many post it gets per month.

double-fault commented 7 years ago

ok. I'll look into it on the weekend when I get time.

NobodyNada commented 7 years ago

I'll look into that when I have time as well. For now, I'll adjust the report threshold in SOCVR. About how many reports would you like? 1/2 the amount that's posted now? 3/4?

NathanOliver1 commented 7 years ago

@NobodyNada If we could do half for right now and then we can open that back up once the changes happen that would be awesome.

double-fault commented 7 years ago

I just looked a bit into this today again as I finally got some time. Sorry for the late response. Do note that @NobodyNada will have to implement this (not me) as I don't know Swift, so he will also have to be available for this.


I went and checked the api if it was query-able for the tags and their question counts, and it appears it can be done. The count returns the total questions in that tag.

I checked it, and it appears that the last page is page #482.

There are two ways in which we can define this predefined list: one if the tag is in page < x, or if the count is smaller than y. IMO, checking which page the tag is better than checking the count. But both ways should work.

We will even have to decided which page and/or count we should check till. I guess we should query this once in a month or so otherwise we will get rate-limited by the API.

NobodyNada commented 7 years ago

Thanks, Ashish! I'm thinking the delay should scale based on the tag's popularity. So if it's tagged Java, it will wait 30 minutes, but if it's tagged assembly or something, 5 or 10 minutes.

I don't know when I'll get around to implement it though. My cousin is in the hospital from a skiing accident, so my free time is reduced.

double-fault commented 7 years ago

Sure, take your time. Hope your cousin gets better soon.

Your idea seems to be good, but there are some problems (which will exist with every delay design). Sometimes, the question gets closed fast, so we will have to check using the api again before posting in the room. This means we will need 1 extra api call per post. I'll work on this more whenever I get time again.

NobodyNada commented 7 years ago

Oh yeah, that's a good point. We'll have to make 2 requests per post: one to get the tags and one to get everything else.

double-fault commented 7 years ago

Don't the tags come in the original api call? There is a option in the filter IIRC.

NobodyNada commented 7 years ago

Right; but we have to make two requests because the first one needs to get the tags and the second one needs to get the closure status.

double-fault commented 7 years ago

oh ok then. I misunderstood you in your earlier message then.

double-fault commented 7 years ago

I'll look a bit more into the detail (maybe) tomorrow or next week. I'm doing some more projects right now and my exams are coming in March so I'm kinda busy. Going to sleep now, goodnight!