Feedback: Quiz feature (beta)

[Zarthus]

Post feedback for the quiz in here. Include browser version and OS

[Zarthus]

Improvements

• [x] Change Icon to something more sensible, i.e. https://fontawesome.com/icons/gamepad?style=solid
• [x] instead of setting a fixed height I'd let it automatically expand, so you don't get scrollbars. https://liefland.net/~zarthus/image/chrome_3eadffbb5e.png https://liefland.net/~zarthus/image/chrome_2f7f9a55eef6ea07b5.png
• [ ] Display the text it is failing to match against, while also posting the result you currently have. This makes it more visual where you are going wrong (forgetting the g flag for instance).
• [ ] Hide the hints by default / divide them in a different section.
• [x] Mark the engine used. Not everyone is familiar with PCRE and the featureset.
• [x] This is very confusing to me, please only has one "A": https://liefland.net/~zarthus/image/chrome_90d94dd8ac.png

Bugs

• [x] Not parsed by markdown properly: Test 3/8: Hmm. Try to make your pattern match T, at least. Check out (http://www.regular-expressions.info/charclass.html)[http://www.regular-expressions.info/charclass.html].

• [x] Switching between tasks, it remembered the error but not the regex: https://liefland.net/~zarthus/image/chrome_49378fe8.png

Difficulty Curve

• [x] 4 is extremely simple, matching numbers is something I learned way before learning about word boundaries.

• [x] Wrong / misleading error: https://liefland.net/~zarthus/image/chrome_eee8bd16.png

• [x] Consider different wording to make challenges to understand (or do feature "Display the text it is failing to [...]")

  • One way of phrasing a challenge different could be: "Validate an IPv4 Address. IPv4 addresses can only have a maximum of 255 in one octet, cannot have leading zeroes, and cannot be zero in the first octet."

  • The original is "Check if an IP address is valid. An IP address is the numerical label assigned to any device connected to the Internet. In this task you are to validate a standard IPv4 address. Start by trying to validate 172.16.254.1."

Windows 10, Google Chrome Version 71.0.3578.98 (Official Build) (64-bit)

[Timvde]

Regarding the bug: correct markdown is [text](url), not the other way around.

Improvements

• [ ] Keep a history (could be in localstorage if you don't want it serverside)
• [ ] Prefill your current attempt in the editor if you navigate to there from the quiz
• [ ] Maybe also the other way around, after writing your regex in the editor? If you're overwriting previous attempts, the history is really important though.
• [ ] Show a live counter with the current length of the regex you're editing

Bugs

• [x] Sometimes (not consistently reproducable) the last attempt disappears when navigating away (e.g. to the editor) and back
• [x] Solve a task; see the new task get unlocked; submit a wrong entry for the task you just solved; navigate away and back; new task is now unavailable
• [x] Task 18: Task description says **, solution needs a single *.

Typos

• [x] Task 16: Test 8/32: You're regex is case sensitive...? | sed "s/You're/Your/"
• [ ] Task 16: Test 18/22: You should not use backreferences, unless there's nothing that needs to be retrieved. Use (?:group) instead of (group)`: too many negatives that break your sentence.
• [ ] Task 14: In the task description, use https instead of http for the URL

Arch Linux, Firefox 64 (64 bit)

[firasdib]

Improvements

• [x] Show a leaderboard -- should this be on each task, or one condensed page with this info? What about privacy?
  • Zarthus/Edit: Consider an opt in checkbox?
• [x] Save regex attempt locally, in case you switch views without submitting it to the server.
• [x] Save the best (shortest) attempt on the server (to encourage golfing) and show it when you view a task
• [x] The quiz link should automatically return to the previous task you were visiting, not always the latest.
• [x] Add more extensive test types with more granular control

[Ouims]

feedback so far: quiz 2 - test 7/8 says 'every occurence of i' or 'every I' --- I think it should go with the description and talk about 'every occurence of the word i' and there should not be a capital I there. quiz 2 - the shortest solution is without the /i flag, but without the /i flag you're going to match all of the word "I" as well which isn't as efficient, I don't know if that's a concern but it felt wrong when it told me i didn't have the shortest solution. note: this isn't about efficiency i guess.

quiz 6 - without @firasdib telling me what i should be doing i'd never have understand it's one key press generating two extra key, i'd suggest using that word 'extra' or finding better wording overall

[jay-tea]

I'd recommend the following fail test for task 7: "1/1/1/1"

[TWiStErRob]

Android Chrome and Chrome on Windows 10:

Intro:

• [x] Fix grammar with: s/\.([A-Z])/. \1/
• [x] Add "let's go" button to bottom (on phone you have to open the sidebar to go to next)

General:

• [x] When I'm done with a task, there's no way to go to the next one, replace the submit button with "Next", but if the regex is edited, change it back to "Submit"
• [x] When I press "Submit" and response is slow (the server is down or I don't have internet or it is slow) the previous error message shows up while the progress indicator is shown.
• [x] Would it be possible to have it work offline?

Other (unrelated to quiz, but found during)

• [x] in the explanation of (?<!.) does "Assert that the Regex below does not match" supposed to be red?
• [x] in the explanation of (?<!) is null intentional?

Q1

Excellent! I guess you have some regex knowledge, so let's get started!

• [x] I already started... also let's get started could be clickable to goto next task.

Q2

Let's test your knowledge

• [x] += " more"

Q3

You can with regex return the number of matches.

• [x] → "You can, with regex, return the number of matches."
• [x] → "With regex you can count the number of matches."

sentence structure was strange + the it doesn't return the matches, it's not a regex feature as far as I know, but there's the API to get all the matches. Your validation script is counting the matches, not the engine, so using "return" is a bit off here. I would re-word as "count" and "match individual characters" instead of "return"

Test 5/8: You seem to be counting vowels. A gentle reminder: aeiou are vowels.

got the above for [A-Z]/g

• [x] No, I'm not counting vowels, it's sounds like I'm only doing vowels. Maybe "You seem to be counting vowels too." or "seem to be including vowels".

Q4

• [ ] again, not counting, but matching all of them

Q4 and Q5

• [ ] these feel a bit out of place, they're too easy compared to the previous ones that needed negative lookahead and understanding boundaries

Q7

Each number should be at most 255.

• [x] This message implies a lot of things, would it be possible to help which number is not matching? I had a hard time noticing that I'm allowing 299.

• [x] an accepted regex matches these as valid: 0.0.0.00 00.0.0.0 00.00.00.00

Q8

Test 11/15: Also remove empty tags: <>.

• [ ] This is taking it a bit far, I thought we're assuming somewhat valid HTML, although it's not a biggie to add <>|, but a bit moot.

Test 12/15: Tags can sometimes span multiple lines. For that reason you should also remove partial tags, which may be missing either < or > if they occur at the start or end of the string, respectively. For example, body>text</body

• [ ] OK...? This is getting a bit frustrating handling edge cases that are not specified in the task. The input string body>text</body does not contain HTML tags, just a > and a < characters!

/<\s*>|<\s*([^\s>]+).*?>((.*)<\s*\/\s*\1\s*>)?|<\s*([^\s>]+).*?\/\s*>/gs actually handles whitespaces in all the places (note the s flag and the use of \s* all over), even where there shouldn't allow (e.g. </ and /> must be without space as far as I remember, maybe in XHTML, but still), here's my test that replaces as expected, yet I get the above error:

<
normal-closed
>
te
xt
<
/
normal-closed
>

<
unclosed-tag
with-attribute
>

<
wrapped-self-closed
with-attribute
/
>

... can't proceed due to this last failure.

• [ ] (Edit) After talking to Firas for a bit and realizing I don't only have to strip valid HTML, but anything that looks like it I simplified it and solved the problem, but the solution strips reasonable text: My age is > yours. -> yours. that doesn't remotely look HTML. I think test 12 is just making it unnecessarily complicated without actually asking for it in the task description.

[TWiStErRob]

General:

• [x] show the percentile along with shortest solution ("you're better than p% of the solutions"); example from leetcode.com

Q7

• [x] task name is lowercase, any chance of showing IP? or at least change to ip address

Q9

• [ ] Explain what the email parts are in the question. <local-part>@<domain>.<top-level-domain> would be helpful in understanding the test failure messages, not everyone is familiar with the RFC terminology.

Test 16/44: The pieces of a domain must not end with a hyphen.

• [ ] This is where "simplified" doesn't feel true any more.

Test 22/44: The local-part is now allowed to end with a dot, for example, x. is not an allowed local-part.

• [x] What? The beginning and end contradict each other. Did you mean s/now/not/?

Test 29/44: @ must not be allowed in the local-part or domain. Test 30/44: ()<>,;:\"[] must not be allowed in the local-part.

• [ ] ok, that's it, this is ridiculous; this "simplified" task is taking as long as all the others before.
  @twisterrob goes to https://emailregex.com and copy-pastes ... "General Email Regex (RFC 5322 Official Standard)" doesn't pass the test, ouch

Q10

set backreferences with a, c, and -

• [x] match a, c, and - into match group 1

For every occurence of the char #, backreference the previous character. Example: for the text "a#bc# -#", set backreferences with a, c and -. You are not allowed to consume the hash character.

• [x] Formatting is a bit strange, consume is a keyword for some reason, but #, a, c and - is not monospace.

Q11

Now I can effectively match floating point numbers in any given text document!

• [x] "Now you can"?

Q12

• [x] Could have been a great task before the IP validation (Q7)

[TWiStErRob]

7cf2f44 Don't allow anything but letters in the tld - Firas Dib f8cda54 No numbers in TLD - Firas Dib

hmm, this broke my existing accepted submission; after modifying a test case, consider re-running all the submissions and flagging the failing ones on the UI?

Similarly when changing requirements the originally sortest solution may not work any more, and hence it would be impossible for anyone to match the shortest solution length. Do you have anything to prevent this?

[firasdib]

@TWiStErRob No, there is nothing to prevent this. Hopefully those cases will be very limited.

[kepexx]

General: The error messages are very vague, and so are the requirements. Having the actual unit tests that are done on the regex would be helpful

Q7

Only digits are valid in an IP address.

My regex doesn't match non-digits except for the dot separator.

[firasdib]

@FireCubez Thanks for the feedback.

The general idea for the quiz is to not give away any solutions or intricate details, and that the user has to figure out exactly what they need to do. They need to read the task carefully and listen to the error messages. It feels more like a challenge that way, otherwise it's just a regular task. The feedback you get is meant to guide you somewhat, and is more helpful in the earlier tasks than the later.

However, if some tasks have insufficient information in the task description, or some tests are unclear, these need to be addresses.

My regex doesn't match non-digits except for the dot separator.

If you get that error, I guarantee you're matching non-digits.

[Zarthus]

• [ ] a graph of "people who completed Q1 Q2 Q3 Q4 ..."
• [ ] include number of people in existing completion graph. not just %

[TWiStErRob]

@Zarthus I expect curious to see if it's true

[Zarthus]

I expect :P

[firasdib]

@TWiStErRob @Zarthus Both of those graphs are, quite likely, the correct approximation. Is it any useful or fun info to share? If so, where would it be placed?

[Zarthus]

@firasdib I'd introduce a separate menu in the very bottom with "usage statistics" to not discourage people who land in "Introduction". Perhaps gate it off to 5 quiz completions to avoid people from being discouraged right off the get go.

[firasdib]

@Zarthus Would that be a full page with a single graph in it?

[TWiStErRob]

Agree with @Zarthus + other than the graph, in the future,

• [ ] you could also do some nerdy stats in a table, like number of submissions tried/succeeded, fastest solution, success with most tries, time between first and last submission (could be tricky if you don't have tracking) etc.

[knoebber]

Task 9 - test 30 and 31 have same error message: "Whitespace must not be allowed in the local-part."

[firasdib]

@knoebber Thanks for the feedback. Multiple tests have similar/same messages, which is by design. Typically its not an oversight, but rather multiple tests trying to ensure you're not cheating in any way :-)

[Cosmatevs]

Task 7: It says "The addresses can only have a maximum value of 255 in either octet, and cannot have leading zeroes" but it accepts solutions that capture two- and three-digit numbers with leading zeroes, e.g. 01.001.01.001 Edit: Yeah, I meant task 7, not 8.

[firasdib]

@Cosmatevs Do you mean task 7? We used to require not having leading zeroes, but this complicates an early task too much, and was thus removed. I'll correct the text.

[Cosmatevs]

Task 20 (quoted text with escapes): Solutions that capture multi-line quotes are accepted, e.g. "quo te" In addition, surprisingly, global or multi line flags are not required.

[firasdib]

@Cosmatevs Since you need to match a single string in quotes, you don't need /g, as there is only one match possible. Consequently, the /m flag is unnecessary. Multi-line quotes are perfectly fine.

[Ouims]

On task 3, if you match non printable character, the regex is accepted, but these char are ascii, so it's very misleading to say 'ascii' in the description i think. It's especially a problem if you want to find out how to get the best score 16 char.

I think the 16 char entries should be invalid with the best score being 18.

[Davidebyzero]

On task 3, if you match non printable character, the regex is accepted, but these char are ascii, so it's very misleading to say 'ascii' in the description i think.

I would hold that the 16 char solution is valid, because the problem's input domain does not include nonprintable ASCII.

[Ouims]

yeah it does, it's possible to enter ascii char in there. I successfuly entered ascii 1, so it's possible to handle all ascii with ascii 0 being impossible to handle there.

[Davidebyzero]

yeah it does, it's possible to enter ascii char in there. I successfuly entered ascii 1, so it's possible to handle all ascii with ascii 0 being impossible to handle there.

This only proves that a 16 character solution is valid, even if including nonprintable ASCII as you want it to.

Most software can't have ascii 0 inside a string, due to using null-terminated strings.

[Davidebyzero]

Task 7: It says "The addresses can only have a maximum value of 255 in either octet, and cannot have leading zeroes" but it accepts solutions that capture two- and three-digit numbers with leading zeroes, e.g. 01.001.01.001

I don't consider that to be a problem, as an IP address with leading zeroes in some of its octets is still entirely valid. For example, 127.000.000.001 acts the same as 127.0.0.1.

Also, the problem does not say anywhere that they cannot have leading zeros. Unless you're talking about a particular "Test n/107: ..." error message you saw?

[Davidebyzero]

On task 23, a regex that changes ]* to ], or ]*c*d[*asdf**]a*f*[sf*f*s to ]cd[asdf]a*f*[sffs, or for that matter erases the *s between the first ] and the first [ in any string starting with ], incorrectly succeeds. I temporarily had such an invalid solution as I was golfing down my solution. (Now it's valid, and even shorter.)

[firasdib]

Task 7: It says "The addresses can only have a maximum value of 255 in either octet, and cannot have leading zeroes" but it accepts solutions that capture two- and three-digit numbers with leading zeroes, e.g. 01.001.01.001

I don't consider that to be a problem, as an IP address with leading zeroes in some of its octets is still entirely valid. For example, 127.000.000.001 acts the same as 127.0.0.1.

Also, the problem does not say anywhere that they cannot have leading zeros. Unless you're talking about a particular "Test n/107: ..." error message you saw?

Initially the task disallowed leading zeros. But I thought this ruined the general experience of this early task and complicated more than necessary, and decided to remove it. Some remnants of it remained, but it has been leaned up now.

On task 23, a regex that changes ] to ], or ]cd[asdf*]af[sffs to ]cd[asdf]af[sffs, or for that matter erases the s between the first ] and the first [ in any string starting with ], incorrectly succeeds. I temporarily had such an invalid solution as I was golfing down my solution. (Now it's valid, and even shorter.)

Thanks. I'll fix.

[firasdib]

I'm considering deleting the email task, as I do not believe it adds anything of value to the quiz, and is just "more of the same". Thoughts?

[jay-tea]

I'm considering deleting the email task, as I do not believe it adds anything of value to the quiz, and is just "more of the same". Thoughts?

I'd say it's one of the more interesting ones to shorten, but I'm biased :)

[firasdib]

I'd say it's one of the more interesting ones to shorten, but I'm biased :)

Most of the solutions allow for quite invalid input. The testing isn't far as extensive as it has to be. If I make it stricter, that'll also make it more boring.

Unsure.

[knoebber]

I'm considering deleting the email task, as I do not believe it adds anything of value to the quiz, and is just "more of the same". Thoughts?

I had a lot fun doing that one and learned a lot. I understand that most of our solutions are far away from recognizing all valid emails, but I still think its a valuable exercise.

[firasdib]

Okay, in that case I could add a few more tests to root out the obviously invalid emails, and keep the task.

[Cosmatevs]

Clicking the submit button should force checking if the expression has been changed.

[firasdib]

This issue is getting hard to track, so if you don't mind, would you please open specific issues using the new Quiz Template, so that I can get a better idea of what needs fixing?

Thanks for all the reports guys.

@Cosmatevs Great issue. Very weird though. Could you please open a new one for this?