coursera / coursera-labs

Apache License 2.0
17 stars 23 forks source link

Unable to specify lab as grader of it's own programming assignment. #43

Open MantiMantilla opened 1 year ago

MantiMantilla commented 1 year ago

I'm trying to setup an image for a programming assignment based on the nbgrader context with minor changes like python version, libmamba, jupyter lab and a more recent nbgrader version along with some other packages that can't be installed through the expected means (conda, even after enabling libmamba, can't resolve the environment because of upstream constraints).

So far, building the image works fine and I can create labs based on it which are fully operational with the formgrader. I can also create assignments linked to a lab from this image which students can access, work on, and submit.

The trouble I'm facing though is the matter of the auto-grader. Unless the assignment is linked to a default nbgrader image, it asks for a Docker grader (see fig. 1 and 2) and the drop-down list is empty (I would expect this to show the available labs from the selected image). I also can't just load the files from the same context used for the custom image as that would lack the files necessary to grade the student submission.

The documentation for the nbgrader context specifies Coursera labs nbgrader images are pulled onto both labs and grid platform. The same image is used for learners to solve jupyter notebook assignment and auto-grade learner assignments., which is why I would like to know the correct way to reference the lab as a grader.

TL;DR. How do I link a custom nbgrader lab to the grID platform?

image

Figure 1: Lab grading form for assignments linked to a custom nbgrader image.

image

Figure 2: Lab grading form for assignments linked to a default nbgrader image.

echuber2 commented 1 year ago

I'm not sure if Coursera monitors issues here. You may want to reach out to your partner support for help. My understanding is that the official Coursera labs are specially made to work as both the lab image and the (implicit) grader image without additional configuration. However, when educators make custom labs, we do need to craft the grader separately. So you may need to re-implement what Coursera (presumably) has done on their end: create a custom grader that unpacks the student code into the folder structure that nbgrader expects, run nbgrader on it to process the grade, and then convert that into a result (either the json alone or the rich HTML feedback). This is what we did at my institution for a custom nbgrader setup.

MantiMantilla commented 1 year ago

I suspected as much and is in fact what I ended up doing. Just wanted to make sure I wasn't missing a simpler solution that would avoid having separate docker builds for lab and grader. I've reached out to Partner support just in case. Might close this issue by the end of the week documenting how to deploy both lab and grader.