Should we revert the partially-applied track anatomy changes?

[coriolinus]

@TheDarkula put some work into the track anatomy project several months ago. However, he also stopped working on that project several months ago, without getting past round 1. This issue considers reverting the following PRs:

https://github.com/exercism/rust/pull/804 https://github.com/exercism/rust/pull/806 https://github.com/exercism/rust/pull/808 https://github.com/exercism/rust/pull/810 https://github.com/exercism/rust/pull/811 https://github.com/exercism/rust/pull/812

The current exercise tree looks like this:

$ bin/configlet tree . --with-difficulty
Tue Sep  3 11:01:00 CEST 2019
Rust
====

core
----
├─ hello-world [1]
│  ├─ leap [1]
│  ├─ raindrops [1]
│  ├─ nth-prime [1]
│  ├─ beer-song [1]
│  ├─ proverb [1]
│  ├─ difference-of-squares [1]
│  ├─ sum-of-multiples [1]
│  ├─ grains [1]
│  ├─ prime-factors [1]
│  └─ armstrong-numbers [1]
│
├─ reverse-string [1]
│
├─ gigasecond [1]
│
├─ bob [1]
│  └─ matching-brackets [1]
│
├─ clock [4]
│  ├─ dot-dsl [4]
│  ├─ simple-linked-list [4]
│  ├─ pascals-triangle [4]
│  ├─ paasio [4]
│  ├─ nucleotide-count [4]
│  ├─ etl [4]
│  ├─ acronym [4]
│  ├─ sieve [4]
│  ├─ rna-transcription [4]
│  ├─ triangle [4]
│  ├─ grade-school [4]
│  ├─ binary-search [4]
│  ├─ robot-simulator [7]
│  ├─ queen-attack [4]
│  ├─ bowling [4]
│  ├─ tournament [4]
│  ├─ alphametics [4]
│  ├─ two-bucket [4]
│  ├─ spiral-matrix [4]
│  ├─ palindrome-products [4]
│  ├─ saddle-points [4]
│  ├─ isogram [4]
│  ├─ say [4]
│  ├─ run-length-encoding [4]
│  ├─ isbn-verifier [4]
│  ├─ perfect-numbers [4]
│  ├─ hamming [4]
│  ├─ scrabble-score [4]
│  ├─ pangram [4]
│  ├─ all-your-base [4]
│  ├─ allergies [4]
│  ├─ variable-length-quantity [4]
│  └─ pig-latin [4]
│
├─ atbash-cipher [4]
│  ├─ crypto-square [4]
│  ├─ rotational-cipher [4]
│  ├─ simple-cipher [4]
│  └─ rail-fence-cipher [4]
│
├─ anagram [4]
│  ├─ protein-translation [7]
│  ├─ robot-name [4]
│  ├─ ocr-numbers [10]
│  └─ react [10]
│
├─ space-age [7]
│  └─ wordy [4]
│
├─ sublist [7]
│  └─ custom-set [4]
│
├─ minesweeper [7]
│  ├─ rectangles [10]
│  └─ circular-buffer [10]
│
├─ luhn [7]
│  ├─ luhn-from [4]
│  ├─ luhn-trait [4]
│  ├─ largest-series-product [4]
│  ├─ word-count [4]
│  ├─ phone-number [4]
│  ├─ diamond [4]
│  ├─ accumulate [4]
│  ├─ fizzy [7]
│  ├─ roman-numerals [4]
│  ├─ pythagorean-triplet [7]
│  ├─ series [1]
│  ├─ collatz-conjecture [1]
│  └─ diffie-hellman [1]
│
├─ parallel-letter-frequency [10]
│
├─ macros [10]
│
├─ poker [10]
│  ├─ grep [7]
│  ├─ scale-generator [7]
│  ├─ decimal [7]
│  ├─ book-store [7]
│  └─ dominoes [10]
│
├─ forth [10]
│  └─ doubly-linked-list [10]

The exercise tree as of 0b79f7648472b1817289bd3e07ce7f4a7a3f81bb, immediately before #804, looked like this:

$ git checkout 0b79f7648472b1817289bd3e07ce7f4a7a3f81bb && bin/configlet tree . --with-difficulty
Tue Sep  3 11:04:26 CEST 2019
Note: checking out '0b79f7648472b1817289bd3e07ce7f4a7a3f81bb'.
<snip>
HEAD is now at 0b79f76 Merge pull request #801 from ZapAnton/readme_submit_section
Rust
====

core
----
├─ hello-world [1]
│  ├─ gigasecond [1]
│  ├─ leap [1]
│  ├─ raindrops [1]
│  ├─ reverse-string [1]
│  ├─ nth-prime [1]
│  ├─ bob [1]
│  ├─ beer-song [1]
│  ├─ proverb [1]
│  ├─ difference-of-squares [1]
│  ├─ sum-of-multiples [1]
│  ├─ grains [1]
│  ├─ prime-factors [1]
│  └─ armstrong-numbers [1]
│
├─ pythagorean-triplet [1]
│  ├─ series [1]
│  ├─ collatz-conjecture [1]
│  └─ diffie-hellman [1]
│
├─ saddle-points [4]
│  ├─ isogram [4]
│  ├─ say [4]
│  ├─ run-length-encoding [4]
│  ├─ isbn-verifier [4]
│  ├─ perfect-numbers [4]
│  ├─ hamming [4]
│  ├─ scrabble-score [4]
│  ├─ pangram [4]
│  ├─ all-your-base [4]
│  ├─ allergies [4]
│  ├─ variable-length-quantity [4]
│  └─ pig-latin [4]
│
├─ clock [4]
│  ├─ dot-dsl [4]
│  ├─ simple-linked-list [4]
│  ├─ pascals-triangle [4]
│  ├─ paasio [4]
│  ├─ nucleotide-count [4]
│  ├─ etl [4]
│  ├─ acronym [4]
│  ├─ sieve [4]
│  ├─ rna-transcription [4]
│  ├─ triangle [4]
│  ├─ grade-school [4]
│  ├─ binary-search [4]
│  ├─ robot-simulator [4]
│  ├─ queen-attack [4]
│  ├─ bowling [4]
│  ├─ tournament [4]
│  ├─ alphametics [4]
│  ├─ two-bucket [4]
│  ├─ spiral-matrix [4]
│  └─ palindrome-products [4]
│
├─ luhn [4]
│  ├─ largest-series-product [4]
│  ├─ word-count [4]
│  ├─ accumulate [4]
│  ├─ roman-numerals [4]
│  ├─ phone-number [4]
│  └─ diamond [4]
│
├─ atbash-cipher [4]
│  ├─ crypto-square [4]
│  ├─ rotational-cipher [4]
│  ├─ simple-cipher [4]
│  └─ rail-fence-cipher [4]
│
├─ bracket-push [4]
│  └─ luhn-from [4]
│
├─ sublist [4]
│  └─ custom-set [4]
│
├─ space-age [4]
│  ├─ luhn-trait [4]
│  └─ wordy [4]
│
├─ macros [4]
│
├─ poker [7]
│  ├─ grep [7]
│  ├─ scale-generator [7]
│  ├─ decimal [7]
│  ├─ book-store [7]
│  └─ dominoes [10]
│
├─ anagram [7]
│  ├─ protein-translation [7]
│  ├─ robot-name [7]
│  ├─ ocr-numbers [10]
│  └─ react [10]
│
├─ minesweeper [10]
│  ├─ rectangles [10]
│  └─ circular-buffer [10]
│
├─ parallel-letter-frequency [10]
│
└─ forth [10]

In my opinion, the older exercise layout was more balanced and assigned more reasonable difficulties to the exercises. However, that's a very subjective judgment.

I'd like to collect opinions from other maintainers and students about which exercise layout was better.

[kytrinyx]

@coriolinus Thanks so much for the suggestions. I took a look at the historical data for the track, and I’ve decided to leave the track as is for now.

The changes that @TheDarkula made were critical in improving mentor participation in the track. Rust is still by far the most problematic track on the site because the jumps in difficulty are very large, and so mentoring it is not fun. Before the changes, we had no mentors as they all quit. Now we have some, but they struggle to keep up still.

We will soon be announcing an Exercism-wide initiative to properly define and design Tracks (we’ve talked a bit about the problems with the lack of clarity of the purpose of Tracks on the Project Specifications repo). Until then the only changes to ordering we want to make are to specifically address bottlenecks identified by data based on activity on the site.

I'm going to go ahead and close this.

[petertseng]

Since the above comment mentioned exercise ordering, that seems to imply that we should continue to consider difficulty changes. So I will give my opinion on the matter of difficulties in this comment.

But I will limit myself to only those exercises that changed since https://github.com/exercism/rust/commit/0b79f7648472b1817289bd3e07ce7f4a7a3f81bb, to save myself time. I have not gone through every exercise on the track to make sure its difficulty is right.

Note that a 4 and a 7 are equivalent on the site (they both show as Medium) so the only reason for us to differentiate between a 4 and a 7 is for our own purposes. Thus I advise not to spend too much time differentiating between a 4 and a 7. But to spend some time differentiating between 1 vs (4 or 7) vs 10.

Before I give my opinion on the matter, I have to report that I am mostly doing these by feel, and maybe the process would be made easier if there were some standards about what the characteristics are of each difficulty. But here we go.

Here are the difficulty changes:

$ diff -u <(ruby ../problem-specifications/difficulties.rb <(git show 0b79f7648472b1817289bd3e07ce7f4a7a3f81bb:config.json)) <(ruby ../problem-specifications/difficulties.rb) 
--- /proc/self/fd/11    2019-09-04 02:58:55.877147895 +0000
+++ /proc/self/fd/12    2019-09-04 02:58:55.877147895 +0000
@@ -3,7 +3,7 @@
 all-your-base difficulty 4
 allergies difficulty 4
 alphametics difficulty 4
-anagram difficulty 7
+anagram difficulty 4
 armstrong-numbers difficulty 1
 atbash-cipher difficulty 4
 beer-song difficulty 1
@@ -11,7 +11,6 @@
 bob difficulty 1
 book-store difficulty 7
 bowling difficulty 4
-bracket-push difficulty 4
 circular-buffer difficulty 10
 clock difficulty 4
 collatz-conjecture difficulty 1
@@ -23,7 +22,9 @@
 diffie-hellman difficulty 1
 dominoes difficulty 10
 dot-dsl difficulty 4
+doubly-linked-list difficulty 10
 etl difficulty 4
+fizzy difficulty 7
 forth difficulty 10
 gigasecond difficulty 1
 grade-school difficulty 4
@@ -36,11 +37,12 @@
 isogram difficulty 4
 largest-series-product difficulty 4
 leap difficulty 1
-luhn difficulty 4
+luhn difficulty 7
 luhn-from difficulty 4
 luhn-trait difficulty 4
-macros difficulty 4
-minesweeper difficulty 10
+macros difficulty 10
+matching-brackets difficulty 1
+minesweeper difficulty 7
 nth-prime difficulty 1
 nucleotide-codons difficulty 0
 nucleotide-count difficulty 4
@@ -53,11 +55,11 @@
 perfect-numbers difficulty 4
 phone-number difficulty 4
 pig-latin difficulty 4
-poker difficulty 7
+poker difficulty 10
 prime-factors difficulty 1
 protein-translation difficulty 7
 proverb difficulty 1
-pythagorean-triplet difficulty 1
+pythagorean-triplet difficulty 7
 queen-attack difficulty 4
 rail-fence-cipher difficulty 4
 raindrops difficulty 1
@@ -65,8 +67,8 @@
 rectangles difficulty 10
 reverse-string difficulty 1
 rna-transcription difficulty 4
-robot-name difficulty 7
-robot-simulator difficulty 4
+robot-name difficulty 4
+robot-simulator difficulty 7
 roman-numerals difficulty 4
 rotational-cipher difficulty 4
 run-length-encoding difficulty 4
@@ -78,9 +80,9 @@
 sieve difficulty 4
 simple-cipher difficulty 4
 simple-linked-list difficulty 4
-space-age difficulty 4
+space-age difficulty 7
 spiral-matrix difficulty 4
-sublist difficulty 4
+sublist difficulty 7
 sum-of-multiples difficulty 1
 tournament difficulty 4
 triangle difficulty 4

To enumerate:

• anagram: Honestly could go either way on this one.
• bracket-push (now matching-brackets): I do think 4 makes more sense than 1, since it may not be immediately obvious what approach to take. The challenge is not always in implementation difficulty!
• luhn: Sure, I think 4 is more appropriate than 7 here. The approach is mostly spelled out in the README and it's just about finding the right pieces to implement it.
• macros: I will refrain from commenting since I have not done this exercise.
• minesweeper: I could go either way.
• poker: I could go either way.
• pythagorean-triplet: Now that https://github.com/exercism/rust/pull/713 added a larger input (30000), performance has become a consideration. I'd suggest 4 here instead of either 1 or 7. It's no longer straightforward so more than 1. But I don't think so hard as to be 7. But I could live with 7.
• robot-name: I actually do think 4 makes more sense than 7 here, but I could live with 7. Approach is already given, student needs to figure out how to use randomness.
• robot-simulator: Seems mostly straightforward, just need to figure out how to model it, so 4 does seem to make sense. Could live with 7.
• space-age: Same as robot-simulator. 4 makes sense. Could live with 7.
• sublist: Could go either way. They have to figure out generics and the approach to take may not be obvious. Especially if they want to do better than O(n * m).

[ghost]

@kytrinyx thank you for creating exercism! It has been a wonderful place for me to keep learning Rust.

Rust is still by far the most problematic track on the site because the jumps in difficulty are very large, and so mentoring it is not fun.

I'm curious about this phenomenon to compare my notes. Is this fleshed out in #809 or anywhere else? No rush as I can also just wait for the formal announcement.

@coriolinus I noticed a similar discussion in #527 so I think I'll comment there with my subjective experience as a learner and mentor on the track. (mostly learner since I just started mentoring :smile: