Multiple exercises/assignments in a single notebook

[marijanbeg]

Very often, in tutorial notebooks, there are exercises allowing students to self-assess their understanding of the material. It would be great if PyBryt would be able to cover this use case. More precisely:

• Allow having multiple exercises/assignments in a single notebook
• Test each exercise solution separately
• Give feedback for each solution separately

It is not easy for me to see what the right user interface would be so that:

• The additional code students have to write (if any) to instruct PyBryt what code belongs to an exercise and what to execute is minimal
• Flexibility in terms of the number of cells used for a solution is allowed (solution is not bound to a particular cell number)
• Solution code is isolated from the rest of the notebook (students are expected to write each solution from scratch and do not reuse variables from the previous parts of the notebook)

Possibly, a similar solution to tracing_on() and tracing_off() functions would be the most elegant solution. This way, a teacher could insert two extra cells (tracing_on('exercise-1.1') and tracing_off('exercise-1.1')) before and after the cell(s) where the solution is expected to be.

[chrispyles]

Hmm, I think this is an interesting idea. One construction I was thinking of would be with a context manager that initializes tracing for code inside of the block, similar to how time complexity annotations will work.

def sort(l):
    return sorted(l)

with pybryt.check("exercise-1.1.pkl"):
    l = [1, 2, 4, 3, 2, -1, -2]
    sorted_l = sort(l)

[chrispyles]

I think that the context manager approach is a bit more elegant, and am currently considering changing the tracing_on, tracing_off construction to also be a context manager.

[ranigb]

I am not sure I understand the problem. If the assignment has three parts, why can't the teacher check for the existence of certain intermediate values in all parts with a single reference implementation as long as the traced values in each part are different?

For example, if in part one the teacher expects to see the intermediate value [1,2,3] and in the second part the value [5,6] then this could be defined in the reference implementation with appropriate error messages. I can see a problem when the teacher would like to check for the same value in 2 different parts or when the teacher is interested in the computational complexity of each part. Are there additional situations you see in which difficulties may arise?

[marijanbeg]

@chrispyles, context managers would be a really elegant solution.

@ranigb, my main concern is about long tutorial notebooks, with a lot of tutorial examples as well as smaller exercises for students. Treating the entire notebook as a single assignment and comparing it to a single reference can give rise to a lot of unexpected problems. I believe that being able to somehow isolate individual student tasks would be helpful. Also, comparing the entire notebook to a single reference is available by default.

[chrispyles]

Opened #49 which implements the context manager discussed here and will close this issue when merged.