Warning flags for reviewers on various entities

[hburn7]

We need a way to flag games which use beatmaps that are only used once throughout the tournament. This is requested by @sourmongoose.

• Identify optimal strategy (is just a boolean flag appropriate @myssto?)

Edit:

This discussion will discuss the inclusion of warning flags that reviewers can look at during the review process. The idea is to get a brief glimpse into issues that do not necessarily warrant rejecting the item outright.

Confirmed flags:

• Match
  • Regex validation on match title
• Game
  • Beatmap played is used only once in the tournament (helps identify warmups)

[myssto]

I feel like there have been a lot of things that we identify for automation checks that result in the pre-rejection of an entity that we disagree on between a lot of people in the team. I'm wondering if we could expand on this and maybe have another flag enum, something like AutomationCheckSuspicions. I'm thinking it could be something that is sort of a half-way point between two points we have now being that if we want to identify something like this it would either have to create an entirely new and specialized property or be added to automation checks as a rejection reason. This way we could add some information about suspicious scenarios in entities that should be looked at more closely without having to necessarily pre-reject them outright.

[hburn7]

I think having another field for flags as needed would be a great idea.

@cytusine0 / @SourMongoose I am curious what your thoughts are on this one. Are there any current rejection reasons that you feel should just be "flags" (alerts) instead? A flag in this case would not automatically mark the item as rejected, but would have some indicator during the review process.

[myssto]

Well it could also apply to things that we've axed from automation checks or that cytusine / sour weren't exactly the most keen on. For example we could reimplement the regex name check on matches, but it could be added to this new flag instead of being a reason to reject the match.

[hburn7]

Yeah that's a great idea. Shall we make a feature tracking issue for this then? (Actually, we can just convert this issue into that)

[myssto]

I'd like to hear cytusine, sour, and alix opinions on this first but I'm on board. Mainly would like to hear as many use cases for this as possible. Mainly because this means that we will have to re-run automation checks on all tournaments now :D

[SourMongoose]

Wouldn't mind a separate "warning" flag, definitely see the regex check failing as part of this new flag then. Is there a full list of rejection reasons somewhere?

[cytusine0]

The main other "useful flags" I can think of are

• map is extremely low / high star rating (say 3 stars away from the median), very short (under 1 minute?), or has other anomalous stats
• (maybe??) game is short a player, but they disconnected while the map was playing? I don't think we want to count those games but maybe there's something we would still do with those stats down the line. Relatedly, would any of these comments / flags ever want to be public to website viewers? I know we want the list of rejected badged tournaments to show up somewhere on the website instead of the current spreadsheet and I just wonder how that interacts with this.

[hburn7]

@cytusine0 First 1 I think is very doable. Perhaps just SR and duration are fine for now. The second one I don't think is very doable or practical. This would be something manual review covers, ideally.

In my head, these flags would be private to reviewers. I am unsure whether marking a tournament as 'verified' should remove these flags.

[hburn7]

Wouldn't mind a separate "warning" flag, definitely see the regex check failing as part of this new flag then. Is there a full list of rejection reasons somewhere?

See here:

• TournamentRejectionReason
• MatchRejectionReason
• GameRejectionReason
• ScoreRejectionReason