One of the main benefits of learning is for users to engage creatively with the exercise sets. Their learning would be improved if they could solve creative uses of the concepts they are learning, and also come up with their own creative exercises.
This feature is designed to engage users by giving them more practice exercises and helping them engage other students with their own creative exercises.
Problem
Currently, the only engagement that users have in their learning journey is submitting challenge exercises for each lesson module. 8-12 challenges per module. Besides this, it is all reading and self paced, self guided exercises. Most students do not do all of the exercises, and new exercises are difficult to add, so content is a bit old. New exercises are added at a rate of a few exercises a year!
Solution
There are two types of student groups in our solution. First is the learning group, second is the mentor group.
Learners Group
When a student starts a module, they could jump right into exercises and get quick feedback by submitting answers or writing code, and getting quick feedback (your answer or implementation is right / wrong). Each exercise should also support a discussion where students could ask questions. This approach has several benefits:
Quick feedback - When users submit a solution, right away they know whether it is correct or wrong. This is a huge improvement over our existing "reading" format where the users could have made a mistake and would never know.
Direct discussion and higher student - student engagement - If a student has a question, they could directly ask on the page, on the specific exercise. Currently, when a student has a question on the exercise section, it is difficult to ask for help. The student would have to share a link to the exercise and ask on discord. And not every student has done the exercise before so a student currently don't get quick support when they ask.
Fast navigation and review - System could keep track of what the users are weak at. Students are able to pickup where they left off.
Sample UI - Note: A exercise could also ask users to submit a function to do X. The check for correctness would then be a test suite instead of a hard-coded string.
When user clicks on View Discussion, the would be taken to a discussion page below. The discussion page mock should have included the original question for students to main context
Notes:
Infinite number of exercises for each section. Some students feel comfortable after solving 3 exercises, some after 100. Up to them!
What happens when users got the question wrong? Shake and red border.
We want to give users an ability to try again but also don't want to stress the student so after they get an answer wrong, there should be a "view solution" button where we give them the answer.
There will be no explanation of the answer because we don't want to add friction when creating exercise problems and we want users to engage with each other. Also, alot of questions will have similar explanations so it will get very repetitive.
Mentors Group
Mentors are users who has completed a module. They could then submit exercise to challenge the future generations of students (which would show up in the prior example UI). Submitting exercises could be fun (and therefore more engaging for the students) and educational in the following ways:
Seeing all the wrong submissions from students: This will help the mentor gain a deeper understanding of where common traps students may fall for, which would help them pay attention to their own code in the future!
Seeing how other students approach their exercises: Gives mentors new perspectives on their exercises.
Observe how impactful their exercises were: Mentors should be able to see how many students decided to "save" their example exercises.
Extra practice writing tests / coming up with impactful exercises
Observe and interact through discussions - By allowing discussions on the exercises, mentors are able to engage more deeply with future students and identify common learning confusion points.
A tab will only show up when users have completed the lesson
Notes
We believe that discussions are important, so to facilitate new students being able to explain concepts to newer students, each question and discussions around that should have a expiration time.
However, some core questions should be immune from expiration time.
Flagged problems: ie a question was posted in JS1 but the question requires knowledge of Arrays (covered in JS2).
Flagged problems are taken out of circulation and put into admin page (maintained by eng team).
Admin should be able to: unflag a problem mark as hidden.
Problem state transitions:
normal → flagged: (when user flags it, the provide a reason).
flagged → removed: Admin could hide the questions from view because of irrelevance or incorrect information
removed → flagged: When user updates the problem (by correcting it)
flagged → normal: With admin approval, problem goes to normal state.
Admin Group
/admin/exercises would display a list of all flagged problems with 3 action buttons
Unflag - puts the exercise back to normal mode
Remove - puts the exercise to removed mode
Metrics for measurement
To measure for improvement over time:
Number of users who complete module / Number of users who submits exercises.
Increase means students found the engagement rewarding / fun. Decrease means there were no incentive to submit exercises, and we have to look into why that is.
Number of questions users answers before completing module.
Increase means users are finding the questions rewarding and worth repeating over and over again.
Decrease means lower quality questions.
Implementation
Database Schema
Considerations
Each exercise needs to be associated with:
associated with a lesson module AND submodule. I.e. Functions section of JS0.
Exercise: We don't have a list of defined submodules / labels in the current lessons table.
associated with a user (creator)
Each exercise has the following resources:
Discussion table
How do we store the question being asked, and its corresponding answers?
Do we want upvotes to highlight the most relevant / educational questions?
Answer (which belongs to a student)
How to we query like... "top 5 answers" for a specific exercise?
Proposed Schema
id, createdAt, and updatedAt will always be present, so omitted from the schema definitions.
module - stores a list of submodules for each lesson
lessonId - foreign key to reference the lesson
name - name of submodule
content - text/blob, will hold markdown string
Example:
lessonId - 3, name - function
exercises - stores exercise exercises
submoduleId - foreign key to submodule
description - markdown string
answer - answer string
testable - boolean to represent whether this is a testable exercise or not
true - when evaluating answer, run the answer against the testStr column
false - string matching against answer column
testStr - test string to be run.
ExerciseInteraction - stores the interactions students have with each exercise
Fields:
exerciseId - foreign key
userId - foreign key
note - note str
If a note is empty for a user, it is by default skipped
flag - string: reason why its flagged. Maybe answer is wrong?
answer - first answer str that user created
isCorrect - boolean
Can user have > 1 interaction with an exercise? No. It does not make sense for users to have multiple notes / flags per exercise
Sample queries:
Select * from ExerciseInteraction where note is not empty and exerciesId = 5;
This will display the # of favourited types
ExerciseQuestion - stores questions that the user asked
Fields:
ExerciseId, userId → foreign keys
title → string
content → markdown string (blob / text?)
ExerciseAnswer - Stores answers that users submitted
Fields:
exerciseQuestionId, userId → foreign keys
content → markdown string
ExerciseDiscussionLikes - stores user likes
The reason why we want to have this table separately instead of putting a column in ExerciseAnswer table is because we eventually want to have a feature where students could see the answers they have "liked" so they could use it as a bookmark for great answers.
Fields:
exerciseQuestionId, exerciseAnswerId, userId
GraphQL Queries
Student Page
On page load, initial request should query for
All modules for lesson
All user question interactions
All questions for each lesson
User Interactions
Queries
Next question → query for next question
Discussion → Pulls discussion and answers for the exercise problem
Intro
Current UI design link: click me
Goals
One of the main benefits of learning is for users to engage creatively with the exercise sets. Their learning would be improved if they could solve creative uses of the concepts they are learning, and also come up with their own creative exercises.
This feature is designed to engage users by giving them more practice exercises and helping them engage other students with their own creative exercises.
Problem
Currently, the only engagement that users have in their learning journey is submitting challenge exercises for each lesson module. 8-12 challenges per module. Besides this, it is all reading and self paced, self guided exercises. Most students do not do all of the exercises, and new exercises are difficult to add, so content is a bit old. New exercises are added at a rate of a few exercises a year!
Solution
There are two types of student groups in our solution. First is the learning group, second is the mentor group.
Learners Group
When a student starts a module, they could jump right into exercises and get quick feedback by submitting answers or writing code, and getting quick feedback (your answer or implementation is right / wrong). Each exercise should also support a discussion where students could ask questions. This approach has several benefits:
Sample UI - Note: A exercise could also ask users to submit a function to do X. The check for correctness would then be a test suite instead of a hard-coded string.
When user clicks on View Discussion, the would be taken to a discussion page below. The discussion page mock should have included the original question for students to main context
Notes:
Mentors Group
Mentors are users who has completed a module. They could then submit exercise to challenge the future generations of students (which would show up in the prior example UI). Submitting exercises could be fun (and therefore more engaging for the students) and educational in the following ways:
A tab will only show up when users have completed the lesson
Notes
Admin Group
/admin/exercises
would display a list of all flagged problems with 3 action buttonsMetrics for measurement
To measure for improvement over time:
Implementation
Database Schema
Considerations
Functions
section of JS0.Proposed Schema
id
,createdAt
, andupdatedAt
will always be present, so omitted from the schema definitions.module - stores a list of submodules for each lesson
function
exercises - stores exercise exercises
ExerciseInteraction - stores the interactions students have with each exercise
ExerciseQuestion - stores questions that the user asked
ExerciseAnswer - Stores answers that users submitted
ExerciseDiscussionLikes - stores user likes
GraphQL Queries
Student Page
On page load, initial request should query for
User Interactions
Mentor Page
On page load, initial request should query for
Admin Page
On page load, initial request should query for all lessons
User Interactions
Launch Dependencies
Rather than launching everything at once, we should incrementally release features so work could be done in parallel.
Databases tables we would end up creating: module, exercises, exerciseInteraction, exerciseQuestions, exerciseAnswer,exerciseDiscussionLikes
Phase 1 - Modules
It takes alot of time to curate exercise content, so phase1 would be optimized for the ability to create module content
https://www.c0d3.com/admin/lessons/js0
Phase 2 - Exercises
Mentor Group
lessons
challenges
review
Edit Lessonscreate
button option to create an exerciseStudent Group
ExerciseInteraction
Phase 3 - Discussion & Stats
ExerciseDiscussionQuestion, ExerciseDiscussionAnswer, ExerciseDiscussionAnswerLikes
Stats to the mentor group page