search

thewca / wca-regulations

Regulations and Guidelines for the World Cube Association.
https://www.worldcubeassociation.org/regulations/
112 stars 65 forks source link

Officially Recognizing 4x4 BLD and 5x5 BLD Mean Of 3 Results #539

Closed Jambrose777 closed 5 years ago

Jambrose777 commented 6 years ago

I would like to propose that 4x4 Blindfolded and 5x5 Blindfolded mean of 3s be an officially recognized ranking for WCA. Achieving such a result is a huge accomplishment that often goes under the radar since WCA does not recognize these results.

Pros

Cons

Community

As seen on a recent Speedsolving thread. 48/57 (84.2%) voted to officially recognize these means. I would also like to see a delegate vote on this issue.

Statistics / Counterarguments to Cons

eventId formatId rounds
444bf 1 65
444bf 2 232
444bf 3 804
555bf 1 67
555bf 2 142
555bf 3 601

And from 2017 Only:

eventId formatId rounds
444bf 1 2
444bf 2 12
444bf 3 208
555bf 1 6
555bf 2 5
555bf 3  168

Relevant Query:

Select eventId, formatId, COUNT(*) as rounds FROM
(
    SELECT * FROM Results Where eventId = '444bf' OR eventId = '555bf'
    Group BY CompetitionId, RoundTypeId, eventId
) a
GROUP BY eventId, formatId
jfly commented 6 years ago

We already award results based on accuracy on a single result

It's not really clear to me what this means. Could you explain?

Jambrose777 commented 6 years ago

We don't allow a competitor to retry an attempt due to their accuracy being off (DNFing) nor do we recognize attempts where only 2 pieces are off. This means that we award the solve for being completely solved (accurate).

pedrosino commented 6 years ago

I'm impressed with the number of best of 3 rounds. I would imagine those would be much lower.

Are you sure that all those are correct? I think some may be product of organizers not setting the proper format on cubecomps and just filling DNF/DNS in.

Jambrose777 commented 6 years ago

That’s hard to tell, sometimes the organizers offered bo3 big bld, but the competitors only completed 1-2 attempts and DNS for the 3rd. But mo3 was still offered.

SAuroux commented 6 years ago

To be honest, I am rather surprised by the number of Bo1/Bo2 rounds. These days, when using cumulative time limits, I don't believe there is any valid reason NOT to use Bo3 for these events.

Laura-O commented 6 years ago

I do not have a strong opinion on this topic, so I am just leaving some quick comments here:

The absolute number, as well as the percentage of competitors with a potential Mo3 result is quite low (~10% or ~6% of all competitors with a successful attempt in the event). Therefore, results lists would be quite short and I do not expect to see them grow significantly when we recognize the Mo3 format. I consider it much easier to get a 3BLD mean - not only because of the event in general but also because of time limits. Competitions with a 3BLD time limit >5 minutes are quite frequent, which gives average competitors enough time for "safety solves". Competitions with a lenient time limit for 4BLD/5BLD are comparably rare.

But let's leave this open to collect some more opinions/stats/etc.

lgarron commented 6 years ago

I support this.

I think @Jambrose777 and @Laura-O have made the relevant points, and I also think that recognizing means worked out well for 3x3x3. It looks the risk of negatively impacting competitions is low (given the stats about how often bo3 is already used, and the availability of a cumulative time limit), and it would be nice to recognize the means as long as such rounds are held.

Out of curiosity, do we have data on how much 3x3x3 BLD accuracy improved after we officially started recognizing mean of 3?

The absolute number, as well as the percentage of competitors with a potential Mo3 result is quite low (~10% or ~6% of all competitors with a successful attempt in the event). Therefore, results lists would be quite short and I do not expect to see them grow significantly when we recognize the Mo3 format.

4BLD/5BLD has always had a low number of competitors, and I think that recognizing means can only help encourage people to work on accuracy on speed to complete means. Given that this takes only a modest effort to support (we already have competitions with the appropriate format, and the WCA website already has similar support for 3BLD means), I think it's a step worth taking.

danielkcsheppard commented 6 years ago

I do not support this.

Blindfolded events is a constant battle between someone's accuracy and speed. We already award results based on accuracy on a single result, and recognizing means will award an even bigger difficulty of accuracy."

I think we reward accuracy enough by simply completing a full 5bld, or doing multi. I think accuracy in multi equates very closely to potential accuracy in bigbld.

Gives competitors an additional goal to strive for in their official results.

This is a very weak argument. There are countless other ways to give additional goals.

These means are already unofficially recognized on WCA as "Missing Averages". They would no longer be missing if they were officially recognized.

Again, I do not think this is an argument for it. You've simply said that it doesn't exist, and if they were added then they would exist.

Adds consistency to our results since 3x3 Blindfolded already has mean of 3 recognized officially.

So we should make everything average of 5 because most events are average of 5? Bigbld is a completely different beast to 3bld. The average success is something like 10 times longer for 5bld.

Most rounds are best of 3, but I believe many of them do not give opportunity for most competitors to do 3 solves, due to cumulative time limits. Stats on this would be useful here. There would be some pressure to offer best of 3 for everyone and bigbld already takes a long time considering how much anyone gets out of a round. A non-DNF 5bld winner or full 4bld podium in a comp is often seen as a successful comp.

lgarron commented 6 years ago

There would be some pressure to offer best of 3 for everyone and bigbld already takes a long time considering how much anyone gets out of a round. A non-DNF 5bld winner or full 4bld podium in a comp is often seen as a successful comp.

But as noted above, best of 3 is already held a vast majority of the time, right? People with a DNF on the first attempt won't have an additional incentive to complete an average, and organizers can use combined bo1/bo3 + a lower cumulative time limit can prevent wasting competition time. Do you see any concrete, significant costs that I'm not seeing?

danielkcsheppard commented 6 years ago

But as noted above, best of 3 is already held a vast majority of the time, right?

But it is offered with time limits that prevent many people from doing 3 attempts I believe. Given how difficult a mean of 3 is, I'd be pretty sad if someone got 2 5bld successes and then they couldn't do a 3rd one due to time limits.

Aside from everything else I mentioned I have one more point: The ranking would not particularly reflect ability. The success rate is so low for 5bld means that you're relying on a lot of luck to get one (yes, I know it's not actually luck whether you succeed or not, but maybe you get the point). Let's say we have 1000 people that are doing a mean of 3 and they all have 21% chance of a success on a given solve. Assuming independence etc, that's 1% chance of getting a successful mean of 3. So they all compete and 10 of them get a mean. Do those people deserve a top 10 ranking while others didn't get that 1% luck? Sure, some people are more accurate than others and deserve recognition, but the success rate is just so low that rankings won't reflect this ability well. Some people are going to disagree with this maths given that I talk about successes as being luck. Think of it more as a hit-rate than luck maybe...

I think the side-recognition they already get is sufficient. I feel that an official ranking would be a little laughable to be brutally honest. It's perfectly worthy of a special achievement award, but is not suitable for a world ranking in my opinion.

xsrvmy commented 6 years ago

Comment on the previous comment: I would say that 1% is an overestimate for a normal success rate of 21%, given how exhaustion and pressure can be a factor on solves 2 and 3.

olfrost commented 6 years ago

Can anyone point me to some discussion about introducing official FMC Mo3? I've seen this previous GitHub issue and skimmed through the Speedsolving post attached.

I'm quite keen to compare the arguments and forum support for introducing them to the current sentiment for 4BLD and 5BLD means, as I suspect the arguments for officially recognising FMC means of 3 are similar to those for recognising big bld means.

lgarron commented 6 years ago

Can anyone point me to some discussion about introducing official FMC Mo3? I've seen this previous GitHub issue and skimmed through the Speedsolving post attached.

That's about it.

I'm quite keen to compare the arguments and forum support for introducing them to the current sentiment for 4BLD and 5BLD means, as I suspect the arguments for officially recognising FMC means of 3 are similar to those for recognising big bld means.

Indeed. FMC mean of 3 was about changing the default format instead of recognizing means for an existing format. The latter is a lot less disruptive (although the former led to lots of FMC competitions, which I think are great).

olfrost commented 6 years ago

Thanks Lucas! My reason for asking was that I don't remember seeing a great deal of discussion around it aside from those two resources - it was a bit of a surprise to me when I saw that they were being introduced. Maybe my memory doesn't serve me well (feel free to correct me) but compared to the amount of discussion and campaigning required to get 3BLD means officially recognised, FMC means had a breeze of a time being made official, despite being a far more disruptive change.

I still don't understand why it's considered such a disruptive thing. Competitions can still place time constraints on competitiors, just like 3BLD is done at UK comps for example, and if you are fast enough and accurate enough to get a Mo3 then you will be rewarded as such. The exact same amount of resources are used whether means are officially recognised or not.

If, as a competitor, my abilities are on the cusp of completing three attempts in a given time constraint, then I should realise that there is a possibility that I might not be able to complete a mean of 3. This is true for every single event with a soft cut.

I also don't understand why success rate really matters - I'm arguing for a consistent set of criteria for what should be considered official and what isn't. The fact that competition resources aren't significantly affected (if at all) by having means recognised officially, that competitors aren't necessarily forced to change their strategy, and that there seems some sort of demand for it (as well as multiple discussions around it on Speedsolving) seems to at least warrant a stronger consideration.

EDIT: Some grammar stuff. I am somewhat biased, of course, as I would stand to gain a world record from this. But I do believe that what I've said is rational and logical.

KitClement commented 6 years ago

Maybe a compromise here considering the time limits often preventing people from completing all three solves is to recognize both full means and 2-means. The "Missing Averages" page currently does this anyways, ranking all people who only have 2-means below those who have a full mean. We could have a little note like the "?" bubble that appears for those who changed countries (like Anthony) that explains the distinction between a real mean and a 2-mean. It won't be a perfect compromise to someone that only gets 2 of 3 solves due to time limit, but they at least get acknowledged, and possibly encouraged to get faster to get all 3 in next time.

lgarron commented 6 years ago

As far as I understand, this proposal is very popular with the community, recognizes real skill, is unlikely to disrupt competitions, and is fairly easy to implement on the website. A similar change for 3x3x3 BLD also went well. I think that's a pretty good deal.

@thewca/wrc-team Any objections? Want to put it to a Delegate vote?

Maybe a compromise here considering the time limits often preventing people from completing all three solves is to recognize both full means and 2-means.

I would prefer the simplicity of only recognizing a single type of mean, i.e. only a single "main format" for an average. But I don't feel strongly, if others want to recognize mean of 2 and implement support on the website.

danielkcsheppard commented 6 years ago

Is this going through then? So far I've seen 2 people supporting it - the Jacob and Lucas. Is there more evidence to the "popular with the community" statement?

If it is going through, do people get silly retroactive records again?

KitClement commented 6 years ago

This poll on Speedsolving seems to indicate fairly wide community support: https://www.speedsolving.com/forum/threads/should-4bld-and-5bld-means-be-recognized-officially.67187/

olfrost commented 6 years ago

@danielkcsheppard I honestly don't understand what you're getting at with that "silly records" comment. Some people consider having FMC averages with decimal values silly.

olfrost commented 6 years ago

In fact, it would be sillier to ignore the faster retroactive records and recognise much slower averages done after the regulation came into effect.

lgarron commented 6 years ago

@thewca/wrc-team Ping; any objections or opinions?

xsrvmy commented 6 years ago

Do most competitors even have a reasonable chance of getting a 5bld mean? I remember Thomas Nelson saying that he gets 50% success rate when going safe. That means he needs 8 competitions to get a mean. Given that 5bld means are held so rarely and cumulative time limit can be a problem, most competitors will never get a chance to complete a 5bld mean.

Incidentally, is there any database information on time limits for 5bld bo3 rounds?

jfly commented 6 years ago

Incidentally, is there any database information on time limits for 5bld bo3 rounds?

Yes there is! (If you're curious what the cumulativeRoundIds below is, see the WCIF documentation here: https://docs.google.com/document/d/1hnzAZizTH0XyGkSYe-PxFL5xpKVWl_cvSdTzlT_kAs8/edit#heading=h.3z7evtp0blyd)

 Showing rows 0 - 24 (26 total, Query took 0.0396 seconds.)

SELECT DISTINCT(time_limit) FROM rounds
JOIN competition_events ON competition_events.id=rounds.competition_event_id
JOIN Competitions ON Competitions.id=competition_events.competition_id
WHERE Competitions.isConfirmed AND competition_events.event_id="555bf"

time_limit  
{"centiseconds":360000,"cumulativeRoundIds":["555bf-r1"]}   
{"centiseconds":360000,"cumulativeRoundIds":[]} 
{"centiseconds":900000,"cumulativeRoundIds":["444bf-r1","555bf-r1"]}    
{"centiseconds":300000,"cumulativeRoundIds":["555bf-r1"]}   
{"centiseconds":630000,"cumulativeRoundIds":["555bf-r1"]}   
NULL    
{"centiseconds":300000,"cumulativeRoundIds":[]} 
{"centiseconds":540000,"cumulativeRoundIds":["555bf-r1"]}   
{"centiseconds":270000,"cumulativeRoundIds":["555bf-r1"]}   
{"centiseconds":720000,"cumulativeRoundIds":["555bf-r1"]}   
{"centiseconds":300000,"cumulativeRoundIds":["555bf-r2"]}   
{"centiseconds":450000,"cumulativeRoundIds":["555bf-r1"]}   
{"centiseconds":120000,"cumulativeRoundIds":[]} 
{"centiseconds":360000,"cumulativeRoundIds":["555bf-r2"]}   
{"centiseconds":180000,"cumulativeRoundIds":["555bf-r1"]}   
{"centiseconds":600000,"cumulativeRoundIds":["555bf-r1"]}   
{"centiseconds":180000,"cumulativeRoundIds":[]} 
{"centiseconds":60000,"cumulativeRoundIds":["555bf-r1"]}    
{"centiseconds":480000,"cumulativeRoundIds":[]} 
{"centiseconds":420000,"cumulativeRoundIds":["555bf-r1"]}   
{"centiseconds":360000,"cumulativeRoundIds":["444bf-r1","555bf-r1"]}    
{"centiseconds":1080000,"cumulativeRoundIds":["555bf-r1"]}  
{"centiseconds":240000,"cumulativeRoundIds":["555bf-r1"]}   
{"centiseconds":210000,"cumulativeRoundIds":["555bf-r1"]}   
{"centiseconds":270000,"cumulativeRoundIds":[]}
lgarron commented 5 years ago

I don't have new points to add, but I'm still in favor of this.

@thewca/wrc-team Any strong opinions at this point? Would anyone like to see a Delegate poll?

Laura-O commented 5 years ago

I would not like a Delegate poll, I think it's essential to have a Delegate poll. This is quite a major change, which seems to come a bit out of nowhere. I can't remember that this was ever discussed on the delegate list.

Additionally, I would like to mention that the votings on the speedsolving forum or on Facebook are not representative. At least I am not surprised that there is support for such a change after it was posted in the Blindsolving Discussion section...

kingmathyall commented 5 years ago

Hi All,

The WRC created a poll voted on by all delegates and below are the results:

Should the WCA officially recognize “Mean of 3” rankings and records based on the times from “Best of 3" rounds for 4x4x4 Blindfolded and 5x5x5 Blindfolded? Yes: 91 votes = 70% No: 39 votes = 30% Totale votes: 130

The WRC will strongly take the results of this poll into consideration when making our decision regarding this change!

kingmathyall commented 5 years ago

The WRC has gone through with the change based on the Delegate poll! Once again, thanks to everyone who participated!

lgarron commented 5 years ago

If it is going through, do people get silly retroactive records again?

Bumping this, since Daniel brought it up via email.

I was not happy with retroactive 3BLD means, but I think at this point we should the precedent that was decided by the Board (i.e. do award retroactive WRs).

@thewca/wrc-team, do any of you think we should not award them?

(Also see https://github.com/thewca/worldcubeassociation.org/issues/3721#issuecomment-453359524)

Jambrose777 commented 5 years ago

I honestly can’t see why this is up for debate. What are the reasons that we should not award them? (Besides labeling them “silly”)

SAuroux commented 5 years ago

Besides: according to today's WCA Motions I am pretty sure that this is a WRT decision.

kingmathyall commented 5 years ago

I see no reason why retroactive records wouldn’t be okay. I think it should be okay!

Sent from my iPhone

On Jan 16, 2019, at 12:26 PM, Sébastien Auroux notifications@github.com wrote:

Besides: according to today's WCA Motions I am pretty sure that this is a WRT decision.

— You are receiving this because you modified the open/close state. Reply to this email directly, view it on GitHub, or mute the thread.

-- World Cube Associationhttps://www.worldcubeassociation.org/ https://www.worldcubeassociation.org/

The content of this email is confidential and intended for the recipients specified in the message only. It is strictly forbidden to share any part of this message with any third party, without a written consent of the sender. If you received this message by mistake, please reply to this message and follow with its deletion, so that we can ensure such a mistake does not occur in the future.

xsrvmy commented 5 years ago

@Jambrose777 I think one possible reason is that not having official 4bld/5bld means affect the strategy of competitors (eg. Stanley).

BTW is it possible to not mark a WR result as WR in the database?

Jambrose777 commented 5 years ago

Yes it is possible, the database doesn't even know a result is a world record unless we assign it.

Competitors still face the strategy dilemma in 3x3 BLD today. Plus, some competitors choose the strategy of going for a mean: https://www.worldcubeassociation.org/persons/2014BOYA01?event=555bf

danielkcsheppard commented 5 years ago

In response to Jacob and Matthew, I could just flip the question round and ask "Why should we award them?" But really, I'd like to offer a bit more thought than that!

Records are a really important part of WCA - the whole idea of having a speedcubing governing body without a good records system would be terrible. World Records are things that should be well-thought-through and awarded when someone has truly achieved something. Retroactive records were set when someone happened to do something that would be defined as a great achievement if they happened now, as (in this case) we decided to make bigbld means official. They were not trying to set a World Record, and nobody else was trying to beat them to the World Record. Sorry, but in my mind they are simply worthy of being a World Record due to this. It's a simple line we can draw. Indeed there are other records you could declare not be "worthy" (e.g. Skewb in early 2014) and if there was an easy solution to that I'd be pushing for it, but there clearly isn't, so there's nothing to do about it. I'd honestly rather have the next first non-retroactive Mean be a WR (no matter how slow) than to recognise retroactive Means. My proposal would be that retroactive times enter the rankings, but no "Records" are awarded for them. i.e. the results count as official, but you don't get rewarded with an Official Record that you never knew you had. The first Mean WR (and CR/NR) would be one that goes to the top of the relevant ranking list.

Random thought: I find it hard to believe that WCA is willing to recognise retroactive means, yet denies World Records that were actually done officially and then beaten the same day (yes I know the arguments that people make against that one).

xsrvmy commented 5 years ago

@danielkcsheppard Is there a reason why you think the new WR does not have to beat the current best mean?

Jambrose777 commented 5 years ago

They were not trying to set a World Record, and nobody else was trying to beat them to the World Record. Sorry, but in my mind they are simply worthy of being a World Record due to this. It's a simple line we can draw.

This is simply not true. Many of the people who have achieved these records were in fact going for records. For others it may not have been their top priority, but it was always a factor in the back of there minds.

Indeed there are other records you could declare not be "worthy"

I’d consider every world record achieved duly worthy. They achieved the time at the date, they have a world record.

Random thought: I find it hard to believe that WCA is willing to recognise retroactive means, yet denies World Records that were actually done officially and then beaten the same day (yes I know the arguments that people make against that one).

As the person who has probably spent the most time researching and investigating all wca records, I can say without a doubt that this would be impossible to achieve fairly, so I’m satisfied with the simplified solution of 1 record a day.

danielkcsheppard commented 5 years ago

This is simply not true. Many of the people who have achieved these records were in fact going for records. For others it may not have been their top priority, but it was always a factor in the back of there minds.

There was no record to get so how can they have been going for the record? It really was not a factor is the vast majority of people's minds. Please provide some evidence that a decent number of people were trying to get means. Sure, a few people were, but nowhere near a significant number.

I’d consider every world record achieved duly worthy. They achieved the time at the date, they have a world record.

The record didn't exist - they didn't get the record. So I should say I have a load of World Records for random stuff I was the first/best to do many years ago? Not really, they were just unofficial stats.

As the person who has probably spent the most time researching and investigating all wca records, I can say without a doubt that this would be impossible to achieve fairly, so I’m satisfied with the simplified solution of 1 record a day.

Fair enough that you think that solution is the best and fairest one. But can you not see the contradictions of fairness in what we're implementing for the 2 issues? One one hand, we're handing out records to people who set times in the past before we decided that the records should exist. On the other issue we refuse to give records to people who clearly set them at the time they did them, just because it might be tough to decide who got it first. p.s. Surely for bigbld we're going to have issues deciding who had records at the end of days, given that in some comps bigbld rounds used to last for 2 days.

danielkcsheppard commented 5 years ago

@danielkcsheppard Is there a reason why you think the new WR does not have to beat the current best mean?

My preference would be to have to beat the current best mean in order to get a record My second preference would be to start from blank records My last preference is retroactive records

pedrosino commented 5 years ago

I have to agree with Daniel that people could have not gone for a record that didn't exist. I'd also prefer to start from blank records, instead of requiring that the current best mean is beat.

xsrvmy commented 5 years ago

Starting with a blank record seems like a bad idea because it might become a pay-to-win system where organizer try to organize 4bld and 5bld right away in an attempt to get a record.

btw is there any debate on whether or not past means will be recognized on the profile?

lgarron commented 5 years ago

p.s. Surely for bigbld we're going to have issues deciding who had records at the end of days, given that in some comps bigbld rounds used to last for 2 days.

Note that the Regs make it clear how to handle this:

9i2) All the results of a round are considered to take place on the last calendar date of the round.

danielkcsheppard commented 5 years ago

p.s. Surely for bigbld we're going to have issues deciding who had records at the end of days, given that in some comps bigbld rounds used to last for 2 days.

Note that the Regs make it clear how to handle this:

9i2) All the results of a round are considered to take place on the last calendar date of the round.

Thanks, forgot about that reg :)

jfly commented 5 years ago

For the record, reg 9i2 doesn't help when determining how to assign records when there are multiday competitions, because (until very recently) we didn't have structured information about the schedules of competitions. In those situations, a human has to go look at schedules and figure out what records to apply. It's an involved, manual process.

abunickabhi commented 5 years ago

I do not think the consistency of having all BLD means will be addressed by having bigBLD mean. I think it will be a bit premature to recognize bigBLD means as official results, as there are talks about have 3BLD as an average of 5. https://www.speedsolving.com/forum/threads/proposal-combined-best-of-3-best-of-5-rounds-for-3bld.69657/ Not enough people have voted on this decision on the SS forum and the decision stands almost equal.

If Combined Best of 3/Best of 5 Rounds is recognized then there will be more inconsistency in the BLD events. Also, there is a chance of MBLD being a mo3 event if the community deliberates and discusses on it anytime in the future.

mark49152 commented 5 years ago

@abunickabhi: Why does it have any bearing on 4/5BLD if 3BLD becomes ao5 in future? It's not a problem that 5x5 is ao5 yet 6x6 is mo3. IMHO this is not relevant.