ppy / osu-web

the browser-facing portion of osu!
https://osu.ppy.sh
GNU Affero General Public License v3.0
980 stars 384 forks source link

Successfully nominating maps that end up ranked should confer benefits #6910

Open Ephemeralis opened 3 years ago

Ephemeralis commented 3 years ago

Presently, there is very little mechanical benefit given to any Beatmap Nominators who successfully contribute ranked maps to the game's body of work via their nomination efforts. Given that the position is entirely voluntary and arguably critical to the game's content generation stream, participating in the nomination process should give some degree of reward to those who assist in it.

These are very basic suggestions to start with:

All of these (bar the leaderboards, I guess) should be relatively simple things that would go a long way in giving BN the acknowledgement they deserve, and also encourage them to not do shonky things with their privileges for shits and giggles since they derive barely anything else from a position that requires hundreds of hours of work to attain and maintain.

Naxesss commented 3 years ago

Will probably need to define what a "successful" nomination is, simply being disqualified doesn't necessarily make it unsuccessful, as the mapper may have just wanted to change something minor, like silencing one slider end.

In https://bn.mappersguild.com/ the NAT associate severity (range 0-3) and obviousness (range 0-2) to each disqualification (see image). I'd consider a successful nomination one which has 0 severity and 0 obviousness, but since osu-web does not have this data that might be difficult to account for.

Ephemeralis commented 3 years ago

Might be worth thinking about porting severity over to osu-web to be tracked properly if it's proven to be an integral part of assessing nominations, though that seems like a topic for another issue.

I'd say whoever scores the final nominations on a set that ends up ranked would be considered as "successful" in this instance.

NoffyNoffeh commented 3 years ago

re the kudosu leaderboards point: this has kinda been brought up in the past about porting the old site general leaderboard to the new one here https://github.com/ppy/osu-web/issues/6228 though it sounds like things related to that would have kudosu rework as a prerequisite.

cl8n commented 3 years ago

All nominators involved in a map successfully added to Ranked should gain a small addition of osu! supporter automatically

rewarding quantity over quality is how you breed BNs and mappers that aim to put absolutely everything in the Ranked section, and they can get away with straddling the line between "garbage" and "just not the best map" because of social status that comes from how this is all set up. it only gets worse when this happens in high volume because the average Ranked map becomes worse and that critical line goes even further down.

this idea gives me the same bad feeling as "Elite Nominator" did except the reward is more immediate and easier to get so it's more of an issue. intentional or not, I think this encourages nominators to rank more maps (with no change to the expected quality), and that's just the last thing we need right now.

rewarding with supporter is a good idea but it should be for actually helping the game/community, not putting any map in ranked. on the other hand, there are plenty of helpful things a BN or even normal modder can do that don't result in a map being ranked, and choosing to not reward those actions skews motivation toward only wanting to rank maps

Replace kudosu leaderboards with a new leaderboard that ranks monthly or seasonal (3 month) tallies of nominators who nominate maps that have: a high number of plays, high numbers of favorites, highest number of overall successful nominations

this sounds pretty difficult to balance in a useful way since farm maps are always going to end up with short-term high plays/favs, but maybe some clever math can save the day

the other suggestions sound good 👍

mzhang28 commented 3 years ago

Isn't there already some kind of method under development (or maybe already implemented) for weighing the difficulty of checking each map for BN evals, so that nominating 4-diff tv size maps aren't worth as much as nominating larger mapsets with more complex elements? We should just use that as a springboard for more nomination-related statistics and rewards.

I think adding the nominators to a beatmap page is valuable but searching for them through the main beatmap search probably isn't. A separate way of searching up BNs is a bit cleaner (like Keitaro's bnstats page)

Also want to stress Naxess's point about disqualifications not always being for serious problems, and that the severity in the BN site should be taken into account when discussing nomination quality.

Naxesss commented 3 years ago

Yeah, I've been developing and testing a way to score nominations after difficulty of checking as you've described, and I believe it's currently the algorithm used by the bnstats page (see https://github.com/rorre/BNStats/blob/master/bnstats/score.py#L53), main aspects being:

Assuming we treat this as activity, someone only touching small sets by experienced/popular mappers would have about as hard of a time to keep up with activity requirements / DQs as someone nominating bigger sets by newer/obscure mappers, is the idea anyway. Obviously it's not perfect but I feel like it's a step in the right direction at least. Although my impression is that other NAT disagree with punishing BNs for low scores, so I'm guessing that rules out treating it as activity.

Another problem is that QA is currently either excluded from scoring or easily exploitable; can just pretend to check sets right before ranked, adding yourself to the card just before it archives, and no one would know (chance of dq at that point is basically 0). If excluded from the score then that would discourage QA, which is what we need to ensure the score and system as a whole even works to begin with, so my concern is that it may do more harm than good to start using it as-is (especially if we reward people getting high scores without considering other aspects the scoring falls short on).

Yusomi commented 3 years ago

ranks monthly or seasonal (3 month) tallies of nominators who nominate maps that have: a high number of plays, high numbers of favorites

I get it's just a suggestion so I am maybe looking too much into it but... this seems to suggest you want mappers to aim for play count and favourites when creating a map, and by doing so you would be discouraging creating maps as a creative outlet or just for fun (since I think it goes without saying the maps topping the play count in the ranked section are usually quite homogenous, and have been for some time). Correct me if I'm wrong but, despite it just being a suggestion, I think your viewpoint on the objective of maps may be visible here. That being that all maps should strive to reel in the most players and keep them playing the game. I can see how that's beneficial from a longevity point of view but I would just like to express that I don't like the extent to which marketability is being favoured over fun/creativity in this proposed system.

I think encouraging a particular behaviour in nomination through reward should be done with caution as giving any type of map a benefit to being nominated over any other type of map has the potential to lead to abuse in the form of maps that are unfavourable to the system being ignored and thus leaving some mappers frustrated.

I do want to say though that I think the overall idea of better rewarding the work put in to nominating is a good one.

Hiviexd commented 3 years ago

soo, is anything gonna happen with this?

peppy commented 3 years ago

If you are going to respond, could you please attempt to make it more useful than what you just posted?

Issues like this are hard to deal with because multiple concepts are all being reported in a single issue. I think a good start would be splitting out tasks into their own issues (as has already been done for #7008, for instance)

Ryuusei-Aika commented 3 years ago

I think it is the time to put this on the table, since the mentality of BNs & mappers (for the latter, most of the time it is related to BNs) is an issue that just keeps worsening (it can be partly observed by checking this thread related to the problematic QAH system) and if nothing is happening on creating a better working environment for BNs, it is only a question of time that very serious problems will occur that could in BNG because voluntary works have limits

NoffyNoffeh commented 3 years ago

Replace kudosu leaderboards with a new leaderboard that ranks monthly or seasonal (3 month) tallies of nominators who nominate maps that have: a high number of plays, high numbers of favorites, highest number of overall successful nominations.

This is something community members have taken to doing themselves, such as https://bnstats.rorre.xyz/score/leaderboard . This is definitely a well wanted feature for friendly competition, the same as for gameplay leaderboards. Even the bare minimum of a kudosu leaderboard is one of the few pages linked from the new website that only exists on the old website.