thewca / wca-regulations

Regulations and Guidelines for the World Cube Association.
https://www.worldcubeassociation.org/regulations/
113 stars 65 forks source link

Scramble Checkers / Require scramblers to sign score cards #214

Closed sarahstrong314 closed 4 years ago

sarahstrong314 commented 10 years ago

This idea was brought up by Raymond Gaslow (Torch) in this thread on Speedsolving.com.

It's a simple idea that would help prevent misscrambles by giving scramblers an incentive to scramble correctly, as well as an opportunity to double check that they're on the correct scramble.

It would also allow delegates and organizers to know which scrambler scrambled each attempt.

lgarron commented 10 years ago

I've discussed this with people in the past, and considered it to be too cumbersome (makes it harder to fined willing scramblers, may not catch all errors – in particular, scrambling the wrong attempt).

But all these cases of people unfairly losing their records make it clear that we need to force some international standard. We currently don't have a way to handle these mistakes, but they are basically 100% preventable if you put resources into it. We should be doing things right. If that means taking more time and doing fewer events, perhaps so be it.

Until Delegates can prove that that they can eliminate this issue through other organizational means, I think it's reasonable to require a scramble checker, or strong requirements like having the scramblers sign off on score cards (assuming the latter is sufficient).

Could someone collect a list of notable cases to lend weight to adding stronger requirements?

sarahstrong314 commented 10 years ago

I don't see how this is any different than how judges have to sign each attempt. I've never encountered anyone who didn't want to judge just because they were afraid that their signatures would hold them accountable for any mistakes.

Besides, if someone doesn't want to scramble just because they would be held accountable for each of the attempts they scramble for, should that person really be scrambling?

I agree that this doesn't prevent every kind of mistake, but it would help encourage scramblers to check more thoroughly. I actually think this would help prevent scrambling the wrong attempt since the scramblers would most likely look at the score card twice: once before scrambling and once afterwards when signing.

Laura-O commented 10 years ago

There is one thing I miss in the whole discussion: If we let the scramblers sign on the scoresheets, we know who scrambled the cube. But what does that mean? What happens when someone loses a record due to a wrong scrambled cube? Is there a penalty for the scrambler?

If so, this would definitely discourage people to scramble. If not, I am not sure if this will really help. I also don't think judges do their job better because they have to sign the scoresheets. It's just the normal procedure they have to follow. So I assume this would be the same with scramblers.

Am Sonntag, 21. September 2014 schrieb Lucas Garron :

I've discussed this with people in the past, and considered it to be too cumbersome (makes it harder to fined willing scramblers, may not catch all errors – in particular, scrambling the wrong attempt).

But all these cases of people unfairly losing their records make it clear that we need to force some international standard. We currently don't have a way to handle these mistakes https://github.com/cubing/wca-documents/issues/121, but they are basically 100% preventable if you put resources into it. We should be doing things right. If that means taking more time and doing fewer events, perhaps so be it.

Until Delegates can prove that that they can eliminate this issue through other organizational means, I think it's reasonable to require a scramble checker, or strong requirements like having the scramblers sign off on score cards (assuming the latter is sufficient).

Could someone collect a list of notable cases to lend weight to adding stronger requirements?

— Reply to this email directly or view it on GitHub https://github.com/cubing/wca-documents/issues/214#issuecomment-56285100 .

sarahstrong314 commented 10 years ago

Judges don't get punished for making mistakes, so I don't think scramblers should be either. If the scrambler makes an honest mistake such as not noticing a corner is twisted, I don't think it makes sense to take any action, let alone punish anyone. If the scrambler continuously makes the same mistake such as scrambling with the wrong orientation, this would allow organizers to give a friendly reminder of what the proper orientations are. If the scrambler is careless and keeps putting down miscrambled cubes knowing they're scrambled wrong, then the organizers would know not to trust this scrambler. Requiring a signature would reduce the third kind of scrambler, though.

On a different note, one additional thing the signatures would be good for would be preventing incidents where someone cheats by putting an already scrambled cube on their scorecard.

lgarron commented 9 years ago

I sent this email to the Delegates list two days ago:

In order to help reduce the need for unfortunate rulings on mis-scrambles, I would like to request that Delegates implement scramble checkers (#214) at WCA competitions

A scramble checker is someone who sits next to scramblers, and:

  1. Does not perform any scrambles.
  2. Carefully checks that each scramble a) is the scramble for the correct attempt, and b) completely matches the scramble image.

In order to test how practical and effective this is, I'd like to request Delegates to try as many of the following bullet points as possible: Use a scramble checker for all rounds, not just final rounds (to test practicality). If there are people available, use a secondary scramble checker to double-check (to see how effective the first scramble checker is). Make sure that the main scramble checker takes it as seriously as if (s)he were the only one. Use the same scramble checker (and secondary scramble checker) for an entire group (to try to detect systematic tendencies to perform certain mis-scrambles). Explicitly caution the scramble checker to check all corners. At some competitions, consider creating a small incentive for finding the score checker finding mis-scrambles (to see how many mistakes we can catch if we're looking for them as hard as possible). Ask scramble checkers to keep a detailed record of all incorrect scrambles. Record the round, group, and attempt number, whether the same mis-scramble was repeated, and if possible, find out what the mistake was (e.g. corner twist, wrong attempt, 4 edges and 4 corners wrong == wrong/missing turn). Try to have the scramble checker mark the score card after checking a scramble (to see if this is not too much of a logistical burden). If you used scramble checkers, please create a separate "Scramble Checking" section for it in the Delegate report (in order to make it easy to collect for information across reports). It would be great to have as much data as possible.

I would love to see that this can solve the majority of our mis-scrambling problem, but I don't want us to mandate anything unless we find that it works well. Since this doesn't require a Regulation change, I hope that some of you can find a way to put this into practice immediately.

I'm hoping this can solve several problems without sacrificing fairness.

lgarron commented 9 years ago

Incidents:

Sydney Spring 2014:

Cubers receiving the same scramble

Unfortunately, this happened on multiple instances throughout the day. When I went to investigate, it turned out there were a lot of younger members scrambling. Whilst I appreciate their enthusiasm, they were not double-checking to see which scramble aligned to which solve. After this happened about 3 times, I then selected a few more experienced scramblers (i.e., experienced competitors) to do the scrambling for rounds or events they were not participating in.

Kjeller Open 2014:

So Jonathan Hamstad (2011HAMS01) solved the first cube in the 3x3 final in 6.506 seconds (OLL-skip), which is very good, and new Norwegian record. You can see the solve here: https://www.youtube.com/watch?v=EeDNnzZ_d6c . Later the organizers asked me to publish the scrambles, and I did. I then asked Jonathan for a reconstruction just to see this awesome solve. He answered that he could not do that, because the scramble was not right. I watched the video again, and he was indeed right, the scramble was wrong. This was two days ago, 5 days after the actual competition.

lgarron commented 9 years ago

PoliMi Italian Open 2014: two repeated scrambles.

lgarron commented 9 years ago

Maranhão Open 2014

I advised scramblers all the time to check very carefully each scramble and it seems to work fine, we had no problem with corner twists or miscrambles. But we still had problems with repeated scrambles. Next time I will advise also about checking always the correct scramble number. I think the double cheking in the scrambles do not spend too much time and is worth it.

lgarron commented 9 years ago

Niddrie 2014

  1. The same competitor received the same scramble twice in a later round and reminded Tim to get experienced scramblers. This could have been avoided with a scramble checker, although I still don't think using experienced competitors is the solution to the problem. (I don't want to dob anyone in, but earlier in the competition an experienced competitor did the same thing while scrambling). Accidents will happen, and attention to detail is the only way to reduce them (without using additional labour).
lgarron commented 9 years ago

The last three examples are all from the last week. Obviously, whatever Delegates/organizers are doing doesn't work. Could we try pushing scramble checkers harder?

lgarron commented 9 years ago

Genius Kid India Open 2014

  1. There were 3 incidents of same scrambles. There was no way of having a scramble checker with the number of reliable people we had (Sorry Lucas). All incidents were made by the same scrambler because of the same reason- he trusted the judge to tell the correct scramble number instead of reading the scorecard himself for which solve was next. So errors were made, but under control.

(Also, that comp had many other staff errors.)

lgarron commented 9 years ago

Austin Cubing Limits 2015

Scrambling mistakes -- I emphasized very heavily to the scramblers that they must check every single one, but we still had 3 or 4 reports of repeat scrambles. A few people, including myself, were given the wrong cubes when competing. These mix-ups were corrected and the cubes were rescrambled when needed. In the future I'll keep a closer eye on who is allowed to scramble -- I've noticed at a few competitions the same 1 or 2 people always seem to be mixing up scrambles and misplacing scorecards.

lgarron commented 9 years ago

CUCEI Championship 2014

We assigned a staff member to be a scramble checker. At first it was very difficult, the scramble checker was slow and this delayed the competition. So I put myself to be a scramble checker too and all went faster. The most efficient way was to only check two adjacent faces and all cornerse, because check the whole cube was very slow. With some practice we could memorize this adjacent faces and the corners and all went very fast.

lgarron commented 9 years ago

Hsinchu Winter Open 2014

To examine which is the better way for checking scramble

system, I did a little experiment on this and the following competition (the report will be finished these days, I’m still working on it). We used a signing system for scrambler this time, and here is the procedure.

We asked the scramblers to sign their name on the left hand side on the score card after they scrambled and checked the cube. This make scramblers to pay attention on scrambling and show responsibility on being a scrambler.

The results are pretty good, I didn’t find any repeated scrambles and wrong scrambles. Some scramblers found them did wrong scrambles before them handing the cube, like they did the third SC rather than the second SC, I asked them to re- scramble the cube, since it is the main reason for repeated scramble. It took us some more time for scrambling puzzles, but I think the time is relative less than using the scramble checker system (this will be described in the next report).

lgarron commented 9 years ago

Huddinge Cube Day 2014

A trustworthy and experienced competitor recognised a repeated scramble on his fifth solve in 2x2x2. I gave him an extra scramble. Most probably, he already got the fifth scramble for his fourth solve, but I did not investigate this further.

lgarron commented 9 years ago

Kaohsiung Open 2014

We set two unexperienced cubers who didn’t participate for being scramble checkers since we thought this is an easy thing to do rather than scrambling and judging. We had two ipad for them to check the scrambles, in order to reduce the fee for printing colored scramblers. The setting is down below, we made them sit next to the scramblers, after the puzzles were scrambled, and they were handed to checkers. After checking the puzzle, the checker will cover the puzzle with covers, and then judges would know that this puzzle was ready to go. The difficult we faced was that some of the puzzles were not easy to recognize (such as pyraminx, the stickers were maybe not the WCA pattern since shengshou pyraminx sticks their stickers in the opposite way). And since I asked them to check all faces pretty carefully, they needed about 20~30 seconds to check the whole puzzle. I think if we are going to use the scramble checker, we need to re-estimate the schedule before the competition. The results were good, only two scramble issues occurred (both were repeated scrambles), but I’m not satisfied with this results, I thought this system should solve all scramble issues, but the result shows that if something is going to happen, it happens. I’ll prefer ask scramblers to sign their name in the next competition.

lgarron commented 9 years ago

Guangzhou New Year's Cube Day 2015

  1. I re-designed the record sheet we used (picture shown below), and I asked all the scramblers to sign or initial names onto the record sheet before they scrambled. (It is different from Hung’s competition in Taiwan, where the scramblers were asked to sign afterwards.) I think this procedure can reduce their mistake possibility, because while they were signing, they were fully aware of the number of attempt and kept it in mind. Eventually, we did not face any repeated scrambles.

unnamed

lgarron commented 9 years ago

Oxford Winter 2014

After the 3x3 final had completed, we were talking about the scrambles and Rob had mentioned he received a 1-move cross on the second scramble. We all did not have that so I showed him the scramble and he confirmed he did not receive that. I awarded an extra scramble. This was our only scramble mistake and the reason was because 3x3 final was the last event of the day, and we were lacking people to scramble, so we unfortunately had a couple of inexperienced scramblers helping us. I reminded them that they needed to check the scrambles but unfortunately it got through somehow. Once we realised the error I notified the scrambler of the importance of checking.

lgarron commented 9 years ago

Ekb Open 2015

At the morning of Sunday we had multiblind. There were 3 competitors: two of them tried to solve 3 cubes and I tried 19. Unfortunately, except us there was only one competitor who could scramble, and he appeared to be not very attentive. After memorizing 14 cubes I found scramble duplicate. Then I stopped my attempt and decided to do extra. Since we planned to start next event in 20 minutes, I tried lesser amount of cubes, generating a new set of scrambles for them.

lgarron commented 9 years ago

Uppsala Open 2015

  • Two cases of repeated scrambles were discovered.
lgarron commented 9 years ago

SESC Santa Amaro 2015

I advised scramblers all the time to check very carefully each scramble, which seemed to work fine as we did not identify any problem with corner twists or miscrambles. But we still had problems with repeated scrambles twice.

lgarron commented 9 years ago

Another Fantastic Michigan Competition 2015 report by Kit Clement:

AJ Blair claimed to receive the same scramble twice in the BLD final. Worst of all, I was the scrambler -- I guess it was a long and tiring day for me. To be safe, I gave him an extra as replacement.

lgarron commented 9 years ago

Frankfurt Cube Days 2015

Given the current discussions about misscrambles, I made sure to have experienced scramblers at any time. Still, there was exactly one scrambling issue during the competition, when a competitor got the 1st scramble again for his 2nd attempt in the second round of 2x2x2. Apparently, that competitor was the first one to do his second attempt in this group and the two scramblers must have rushed too much when trying to launch the group, not noticing there was already one cube brought back after a solve. Of course I asked them to pay more attention, even though I think this happening once at a competition is an understandable human error.

lgarron commented 9 years ago

Olek Gritsenko's report for Ekb Open 2015:

At the morning of Sunday we had multiblind. There were 3 competitors: two of them tried to solve 3 cubes and I tried 19. Unfortunately, except us there was only one competitor who could scramble, and he appeared to be not very attentive. After memorizing 14 cubes I found scramble duplicate. Then I stopped my attempt and decided to do extra. Since we planned to start next event in 20 minutes, I tried lesser amount of cubes, generating a new set of scrambles for them. Another competitor who is very experienced had ended his attempt at this moment, and he scrambled for me. He and the third competitor didn't complain about scramble duplicates in their attempts.

lgarron commented 9 years ago

Finally some good news, in Michael Young's Delegate report for Nub Open 2015:

We implemented signing (we didn't have the staff for scramble checking), which was pretty effective, I think - I definitely caught myself once accidentally giving Matt Bahr scramble 4 instead of scramble 3 in 4x4. (I decided to tell the judge that this was technically the fourth scramble, so please put the score in the fourth line, which worked out.) It took us a while to make into a habit, but after about 5 scrambles of conscious effort, it became automatic. It definitely slowed down the competition a small amount (eg, a scramble for 3x3 might take 12 seconds rather than 10 seconds), but I think it's worth the time.

timhabermaas commented 9 years ago

There was at least one other scrambling incident at Frankfurt Cube Days. A competitor received the fifth scramble for his forth attempt and therefore complained about a duplicated scramble during his (actual) fifth attempt.

I believe most of these incidents will go unnoticed by the delegate since judge and scrambler are usually capable of resolving these mistakes on their own.

lgarron commented 9 years ago

Delegate report for Linkub 2015 by Gunnar Krig:

At one time, when I was overwatching a solve at a solving station, I heard a competitor a few stations away saying to his judge: "That's the same scramble as last solve. I should get a new one". I wasn't able to go over there immediately so I was awaiting being contacted by the judge. When he didn't approach me I walked up to the competitor and ask him what happened. He said he got the same scramble and that he was getting a new one. I then told him that I had to DNF that solve since it wasn't his descision to make, and since the cube was already handed in at the scramblers table I couldn't verify his claim.

Even though I was very adamant about the nescessity of correct scrambling and to carefully verify all colors before handing the puzzles out for solving, there were a few cases of competitors getting the same scrambles as the previous solve. In each case I made sure the competitor hadn't made any turns, took the cube to the scramblers desk and personally verified which scramble it was. Indeed, in all cases the competitor in question was right, and get the right scramble instead. It seems like a certain person was responsible for several of these cases, so we agreed that he didn't do anymore scrambling.

lgarron commented 9 years ago

BOND Open 2015 report by Olivér Perge:

In the first group of Square-1, the 2nd and 3rd scrambles were almost identical; the cubes shapes were exactly the same. One competitor reported a duplicate scramble during inspection. We checked and the scramble was correct, but as he was almost right and honest, I rewarded an extra solve.

During 3x3x3 final, I got the 3rd scramble twice. I warned the judge during inspection, as I was sure it was a mistake. The judge confirmed, so I got the 4th scramble and went on as normal.

lgarron commented 9 years ago

Lifestyle Seasons Summer 2015 Delegate report by Dene:

Dene spent a lot of time at the scramble table (as usual), and otherwise used experienced scramblers for most of the competition. He stressed the importance of checking scrambles, and everyone took the extra time to verify, which was good. We have no evidence of mis-scrambles. However we still had an incident of a competitor getting a repeat scramble during 2x2 which was unfortunate.

lgarron commented 9 years ago

Report of SIGI 2015:

We had two areas. In each area were a scrambler checker. The staff tended to had competitors in the same attempt then the scrambler checker could check 3 o 5 cubes at the same time. This made all events went so fast and smooth.

lgarron commented 9 years ago

St. Francis Open 2015 Delegate report by Not Kevin (Michael) Young:

We used scramble signing, but only started after Pyraminx, because of the slowdown we had due to point 2. However, we still had a couple problems - in one case, a competitor still got a repeat scramble (I forgot to write down who, but it was supposed to be a fourth solve, and the scramble was correct, so my guess is that the third scramble was actually done as the fourth - I gave an extra attempt). A quick observation: the only event that it felt like it really slowed things down was 2x2; to me, most events felt like they weren't much slowed.

And:

The most contentious case was with Skewb, on Nathan Soria's third scramble. I checked the scramble, and I believe that I checked it thoroughly enough - however, when it got to him (using GoPro video confirmation), it was somehow misaligned. As a result, he got a 6-move solution (3-move layer into cancelled hedge) in 1.71. At the time, we thought it was legit, but when he was trying to reconstruct, he found that he wasn't able to; after examining the footage, he found that if he changed the final move of the scramble from a U to a U', he was able to reconstruct it fully. He first presented it as a 7-move solution (3-move layer + hedge = 7), so I e-mailed the Board about what to do. The biggest mistake I made was not making him do a backup extra scramble - at the time, I thought the scramble was still okay, and when one Board member replied with his okay (we hadn't found out it was a 6-move-optimal scramble yet), I moved on. However, once it came up that the scramble was not legitimate, it was too late to have him do an extra, so we had to DNF it in the end.

See #67 for developments about puzzle covers, and see point 4 of this comment on issue #121 for a possible re-solve policy.

lgarron commented 9 years ago

GuateRubik 2015 Report by Adrián Ramírez:

We had a relatively big problem, in the final round of 5x5 the scramblers made a mistake, they didn't change the scrambles to 5x5 first roun, and they kept scrambling 5x5 cubes with scrambles of 4x4 first round (the ones used in the previous round), I noticed, because the first attempt from Mynor Guzman (2014GUZM03) was NR, and also I noticed that the round started very slowly because they were repeating the scrambles. When I realized what was happening, I stopped the round, talked to the competitors that had already tried first attempt, and repeated all the round. Mynor wasn't vey happy with losing his NR, but at the end, he set a lower NR in the last attempt, so everything ended up well. I'll be more severe with scramble checking after this incident. :(

lgarron commented 9 years ago

Hessen Open 2015 report by Sébastien Auroux:

Three cases of repeated scrambles were discovered and the affected (experienced) scramblers were asked to pay more attention.

lgarron commented 9 years ago

New Avge Open 2015 Report from Natán:

  • We had one case of repeated scrambles during 2x2 round 2. But then, two competitors approached me saying they were receiving the same scrambles from round 1. What happened was that the PDF reader on that laptop was crazy: You would open a set of scrambles by double-clicking on it and the PDF reader window would open and display any random PDF opened previously and we would have to look for the right tab. But this time, round 1 was randomly displayed and since it was also 2x2, the scrambler assumed it was the correct set. So, we scrapped the scorecards and restarted that group.
  • We had another case where a competitor claimed a repeated scramble for 3x3 but he did so after solving the cube and getting a worse time than on the first scramble. I refused his claim.
lgarron commented 9 years ago

From the Berkeley Winter 2015 Delegate Report by @jfly:

There was one instance of an incorrect scramble being given (I was given 3x3 round 2 scramble #4 two times in a row). We did not have scramble checkers for this competition. I allowed the Berkeley club to not use scramble checkers because there were so many competitors at the competition.

At the next Berkeley competition, I will make sure they have a more relaxed schedule.

lgarron commented 9 years ago

[Competition Report] MAD Cube Weekend 2015 by Luis J. Iáñez:

We didn't use scramble checkers. I just focused on sitting to scramble only trusted cubers, and reminding them every then and now to check in depth. Only three scrambles were erroneous / repeated. I personally saw one of my trusted cubers solving a couple of 555 and re-scrambling them --I loved the scene. :)

[Clarified later in the thread: the errors were noticed by competitors, not scramblers.)

lgarron commented 9 years ago

Report for MathSoc Open 2015 by Akash Rupela:

The biggest incident , During multi BLD attempt 2, the organiser said that he would scramble and that I should solve ( normally i scramble and dont take part because its the first event of day where barely any scrambler is available and I dont trust other people to scramble quickly and prevent delay during multi) . 30 minutes after Shivam Bansal (2011BANS02)'s attempt started, he called me and showed me that 2 of his cubes were scrambled the same. I immediately took the cube and gave the 13th scramble(he was doing 12) but I am sure it did waste about a minute for Shivam. He did not say he wanted to do a resolve when i asked. I think his result should not be converted to DNF since all he recieved was disadvantage and genuine external assistance. I told the organiser that he should relax a bit as he seemed too tired from day 1.

lgarron commented 9 years ago

Report for Dvina 3.0 2015 by Ilya Tereshko:

We used laptop instead of paper for scrambling. At the end of the first day I forgot switch a group of Pyraminx and competitors of second group made 2-3 attempts before I got it. Scrambler was as a competitor in first group and noticed the same easy scramble and gave me know about it. I switched scrambles and restart this group. Some competitors of second group were unhappy, because they got nice singles. I apologized and they forgave me :-)

(i.e. we need to make sure that scramblers are always using the correct scramble group.)

lgarron commented 9 years ago

There was an extremely bad (and possibly intentional) incident at Phalsbourg Open 2015

In particular, out of the 10 scrambles received by the top two competitors (five each), it appears that 7 were incorrect. The WDC was assigned to investigate; I don't have the results on hand.

Edit (2015-04-18): WDC Announcement

lgarron commented 9 years ago

Report for River Hill Winter 2015 by Felix Lee:

The second round of 3x3 was started with scrambles from heat 5 of 3x3 first round. Evan noticed this partway through the solves. We couldn't only give resolves to the people who were in heat 5 since other competitors were potentially helping during that time, so we had to restart the entire round. The proper scrambles generated for this round were used since they had not yet been opened and nobody had seen them prior.

lgarron commented 9 years ago

Delegate report for Murcia 2015 by Luis:

Nothing special. Just to point something out, I'll mention that, like always, I was very selective with scramblers and became specially insistent with scramble checking. We only had one case of repeated scramble.

lgarron commented 9 years ago

Williams Winter 2015 delegate report by Tim Reynolds:

There was one major incident. During the 3x3 finals, we had 4 competitors solving at a time. They stayed on stage for all 5 solves, and then we brought up the next 4 competitors. First competitors 9-12 went, then 5-8, then 1-4. On the third solve of the last group, Kevin Costello III got a 7.7x solve. Everyone cheered, but he immediately turned to me and said he had gotten the same scramble in the OH final. I had forgotten to switch the scrambles after the last event, and none of the first 8 competitors (including me) had noticed. So I announced that we were starting the finals over with the correct scrambles.

Unfortunately one of the competitors had already left. Fortunately, he had not competed in the OH finals, and so he had not already used these scrambles. I decided that the fairest thing was to keep his results, as he had received no advantage and was unaware of the error when he left. I didn't think it would be fair to him to disqualify his results.

lgarron commented 9 years ago

Delegate Report for Guangzhou More Fun Site 2015 by Ming Zheng:

No repeated scrambles reported, since my staff members are quite familiar with the pre-signing system before scrambling.

(By the way, I'm no longer reporting every incident in this thread. Just ones that I think are more notable. This means there may be even more than this thread suggests.)

lgarron commented 9 years ago

Atlantic Open 2015 Delegate Report by Micah Stairs:

While I had originally intended to try using scramble checkers, I didn't have enough extra people to actually implement it (since we had so many new competitors, we did not have the largest pool of judges to work with). At the beginning of the day, I stressed to the scramblers (which was a team of 8 of the competitors) the importance of ensuring that the scrambles are done properly. So what happened is that everyone always checked their scrambles, and when two scramblers were at one table, they would sometimes check each other's scrambles. We did not have any scrambling issues whatsoever. There were of course many times that a puzzle needed to be solved and re-scrambled when mistakes were made, but we did not have any repeated scrambles or other such problems.

lgarron commented 9 years ago

Delegate Report - Florida Spring 2015 by James LaChance:

3x3 Round 2. [redacted] had been judging all day, except for her group of 3x3. She had asked if she could scramble just for a change of pace (understandably so). The first half of the competitors were already done with solves 2-3 (and 1 competitor had 4 solves done) when she informed me that she had been giving everybody scramble 1, even for their 2nd and 3rd solves. I knew this wasn't the case, as I was in this group, and my 2nd solve was different than the first. She insisted she did. I told her this couldn't be possible as long as she was checking the scrambles properly. She claimed she couldn't have been checking them properly, but again insisted that she was only looking at scramble 1 the entire time. One competitors times did gradually get lower (they had 3 solves done), but several competitors claimed they didn't get the same scramble. This is an ultimate showcase of why scramble checkers are not only helpful, but necessary. I ended up making new scorecards, letting everybody keep their first result, and used a new scrambler from there on using the same scramble set. After talking with Kit Clement, he said I probably should have let them use one of the extra scrambles for their first solve. I agree that this would have been a much better choice. I still have the first set of scorecards for the round in case they need referenced. Please let me know if I made the right call, or if anything needs to change.

lgarron commented 9 years ago

Another incident with very bad scrambles from the SLS Zawiercie 2015 Report by Piotr Kózka:

During the end of the round, a lot of competitors, including the fast and experienced ones, came to me and said, that their cubes were misscrambled. Some of them had wrong scrambles in all 5 solves. And they also told me, that some of the correct scrambles were much easier than the ones they got. We have a lot of good skewb solvers in Poland, many of them at a comparable 5.xx - 6.xx seconds level, and the number of misscrambled cubes could have an impact on the places giving a promotion to the final round. In my opinion it would be unfair to leave it that way and I decided to run the second round of skewb again. This time I divided the competitors into two groups, ensuring that in both of them there are experienced cubers. This time it went very smooth and without any problems. And we were only 15 minutes behind the schedule after the event.

lgarron commented 9 years ago

Tuks Winter 2015 Delegate Report by Donovan Hale:

There were no repeated scrambles since we used a scramble checker. The scramble station was partitioned off well, so we could lay the scrambled cubes out nicely and check for mistakes easily. A few mistakes were picked up and those puzzles were re-scrambled before being taken out to the stations.

lgarron commented 9 years ago

Cube for Cambodia 2015 (Melbourne) Delegates Report by Dene Beardsley:

As stated above we used scramble checkers throughout the day. A few of our more experienced competitors didn't take to it too well, but we basically had to tell them to suck it up. Tim tried to compile some data throughout the day on the number of mis-scrambles caught by the checker, but because he didn't tell anyone what he was up a lot of the numbers are rough estimates: Pyra round 1: 12/300 = 4% mis-scrambled 5x5 round 1: 3/80 = 3.75% 4x4 round 1: 3/129 = 2.3% 3x3 round 1: 6/435 = 1.4% Skb round 1: 1/148 = 0.7% 2x2 round 1: 1/385 = 0.3% 2x2 round 2: 2/120 = 1.7% Overall, not a lot of mis-scrambles, but potentially a few issues avoided. We didn't hear of any issues throughout the day of repeat scrambles or mis-scrambles going out to competitors.

More details from Tim McMahon:

It's good to find that the number of miscrambles was low (0.3-4%). The scramble checkers for some of the second rounds and finals didn't record the number of miscrambles. Hopefully there were none, but I'm pretty sure that there were some given the number of miscrambles that were observed in earlier rounds.

I'd recommend having twice as many "trustworthy" scramble checkers as scramblers to avoid impacting the schedule. Some scramblers saw the scramble checker role as redundant, took it as a personal insult that anyone would check their work, and went as far as deliberately miscrambling during 2x2 Round 2 to see whether the checker would pick it up. One also tried to bypass the checker by placing scrambled puzzles back in a cover for a Runner to pick up.

Some scramblers insisted on lining up all the puzzles with the same scramble and checking them all at the same time. This worked well for the 1st scramble. It became problematic and annoying when scramblers insisted and whinged about the time spent checking 2-3 scrambled puzzles individually for the 2nd, 3rd, 4th and 5th scrambles instead of all at the same time. Competitors were up to different scrambles and I had to check the scorecard to make sure that the scrambler didn't accidentally perform the wrong scramble and then try to convince me to hide the mistake by grouping them together (e.g. Feliks is on the 2nd scramble, Dene is on 3rd, Ethan is on 2nd, but all are scrambled with the 2nd scramble because the scrambler wasn't paying attention and accidentally scrambled Dene's with the 2nd scramble again). There's room for improvement with the scramble checker role (e.g. practicing checking to build up the competency, as opposed to just placing a puzzle in a cover and assuming it looks correct after a quick glance).

lgarron commented 9 years ago

From the Tuks Winter 2015 Delegate Report by Donovan Hale

There were no repeated scrambles since we used a scramble checker. The scramble station was partitioned off well, so we could lay the scrambled cubes out nicely and check for mistakes easily. A few mistakes were picked up and those puzzles were re-scrambled before being taken out to the stations.

Laura-O commented 8 years ago

Something that hasn't been mentioned here is the use of a scrambler list, which is created before the competition and defines the scramblers for every single round and group. Sébastien and me used such a list for several German competitions with a very positive outcome. As a result, other delegates have started to use scrambler lists as well. Besides the fact that this assures that the scramblers are trustable, I also think it motivates many competitors. They see it as an honor to be on this list, so they want to do their job well.

So, as the use of scrambler checkers is not always assuring correctly scrambled cubes, this is another simple and easy to implement alternative.