iHiD
commented
7 years ago @Insti Thanks for opening.
So this is probably the biggest and maybe most controversial change we've made to Exercism and, as you can imagine, it's been one of the biggest areas of thought and discussion for us. It's really important for me to emphasise at the beginning of this post that this hasn't been a decision we've taken lightly. We've put literally hundreds of hours into surveying people, reading research into learning behaviours and reward mechanisms, and debating and discussing this topic. My opinions and thoughts on this have evolved dramatically over the last 8 months, and if you'd told me at the beginning that I would be defending the position of "you can't do exercises until a mentor says so" I probably wouldn't have believed you.
Our implementation decision around this point might be wrong, and there are various other different approaches we could have taken, and I am always open to discussing new ideas and thoughts. However, I strongly feel like this discussion must be framed in the understand of what we are trying to achieve with Exercism, not what it has been like previously, and not what you and I as individuals want. It is important to remember that we are experienced, top-notch programmers, who are faster and more knowledgable than the vast majority of others who are using Exercism to learn, and it is critical that we design Exercism for those who need it the most, not just for ourselves. I'd ask everyone to keep that in their minds as we discuss this.
This post is an overview of the journey we have gone on over the last 8 months, and why we've ended up where we are.
The only thing I'd ask (for anyone that gets involved in this thread) is that you understand that these positions are thought out and that this is a deeply complex issue where gut feelings are problematic and open mindedness and reason are essential. This is the meatiest question around product design that I've faced in 10yrs of doing this.
When reconceiving Exercism, our starting point was our values, mission statement and principles. There are two key points in here that everything in this discussion derive from:
- Exercism’s value is in the community, the discussion and the learning.
- Exercism focusses on the learning journey, not the destination.
Exercism is about learning through [practice and code review], not about learning through [practice or code review]. In exercism previously, code review was a second-class citizen. You could work through lots of exercises, and ignore the mentor side of things all together. We felt strongly that that was wrong and that mentorship and discussion was what made Exercism a learning site, not a "code-golf" site - that this discussion is the most valuable way Exercism help someone learn. In our document on Emotional Stages and in referencing this Viking school post we emphasised that Exercism's place in the learning journey is around supporting people through mentorship. There are dozens of courses that take you through exercises, but that Exercism stands in a unique position of helping people through providing code-review, mentorship and support.
A critical part of learning is reward. Everything we do is reward-driven. Our brains learn through Dopamine. When designing any product, it is therefore crucial that one decides where rewards occurs in order to motivate people the most. In Exercism we could choose to either reward on submission of code, or at the end of a positive mentor discussion. We decided that "completing an exercise" is not about making the tests pass and uploading it - completion is about learning something new through discussion and review with your mentors. Once you've achieved this (i.e. mentor signoff) you are then rewarded, and the method of reward is to unlock more exercises. We talked about this unlocking mechansim a lot in the progression document.
Now, our one big concern from day one has been that this blocks people from spending a couple of hours on exercism attacking different problems. That was a huge problem for me at the beginning and I was very against this whole idea because of it. The way we decided to mitigate that problem (and what changed my mind) was to make sure that tracks were designed in a way that means there should always be side exercises available. So take the C# track for example (as @ErikSchierboom is a maintainer). When you start that you have one core exercise and 9 side exercises available from the start. So you have 10 exercises worth of work to do. Then completing the next core will unlock another wad of side-exercises. So at any given time there should be loads of different exercises available to work through. Getting this track design right is therefore absolutely critical, and will take time to perfect.
One piece of feedback that has been really valuable in this is the that it's not clear that there are multiple exercises available, as people tend to only see the next one in the core-track that's locked and not scroll down to the side exercises. We'll definitely consider how we can make this more clear on the "track progress page" and when we work that out, I suspect it might nullify some of the issues around this.
So that's my quick overview of how and why we've got here. I still worry about the whole blocking people thing - I have done for months - but I feel that for the 99% of people who use Exercism it won't be a problem as they're never going to get through 5-10 exercises before a mentor approves the first. I really want to try this with real users (as opposed to expert maintainers) and see what happens. If it turns out that those "real users" are hitting the same wall, then we will have to think about how we can tweak or change things to avoid this, but so much thought and effort has gone into this (and the discord it has struck was not unexpected) that unless someone can raise a new argument or POV that we've not debated previously, I feel like we should test this path out before rolling back our thinking.
To sum up in one sentence: We want to ensure that people using Exercism are actually making progress in their learning, not just ticking the boxes of making the tests pass. This is the way we feel is best to achieve that goal.
BethanyG
kytrinyx
ErikSchierboom
medwards1771
ilya-khadykin
tleen
NobbZ
hoovercj
ghost
Issue created at suggestion of @iHiD
I very strongly dislike that you need mentor "approval" to move to the next problem.
I'm trying to use Nextercism to learn a new language, but my enthusiastic progress has hit a brick wall because I cannot continue to the next problem.
@ErikSchierboom wrote: