Open adhameer opened 5 years ago
Dear @adhameer, @hannelita, @yurug, and all people interested in this feature,
FYI this use case (multi-part exercises) appears to be quite close to the use case #395 (dual-grader exercise), so @YoanwM and I are considering to devise an implementation that addresses both use cases:
exodir
" specification of learn-ocaml 0.12.So we may post in this issue (or in #395) more details in the upcoming weeks, but the overall idea is as follows:
exercises/index.json
;exo1/part1/{descr.md, test.ml, …}
),exo1/part1/meta.json
won't be taken into account,exo1/subindex.json
file that will specify in particular:
exo1
(difficulty, etc.)learn-ocaml build
time), with the previous partn-1/test.ml → which makes a lot of sense if the successive parts are tightly related, or even are a refinement of the grader for the same exercise; but this setting would be optional − and enabling it for each subpart would just induce a O(2⋅n)−like slowdown for learn-ocaml build
[not O(n²)].(As an aside, the runnning example (exercises involving modules
) in @adhameer's post does not necessarily require some multi-part exercise support: it appears we can implement a single grader with independent tests, for dependent modules/functors… but indeed, it'll be much more natural to benefit from this multi-part feature to achieve this :)
Hi @yurug @YoanwM, following our last video meeting and for the record, here is the recap of the format we finally proposed for subindex.json
:relieved:
subindex.json
:<directory> = <string>
<meta> = {
"kind" : ( "exercise" | "problem" | "project" ),
"stars" : [1 .. 5],
"title" : "Title of the multi-part exercise",
/* In an exercise repository, each exercise must have a unique identifier. */
"identifier" : "some_unique_identifier",
/* Authors with their emails */
"authors" : [["First Last", "first.last@example.com"], ...],
/* The skills and concepts that are practiced by this exercise. */
"focus" : ["skill1", ..., "skillN", ..., "concept1", ..., "conceptM"],
/* The skills and concepts that are required to do this exercise. */
"requirements" : ["skill1", ..., "skillN", ..., "concept1", ..., "conceptM"],
/* The suggested exercises in case of success, RELATIVE TO THE REPO ROOT */
"forward_exercises" : [ "exercise1", "exercise2", ... ],
/* The suggested exercises in case of difficulty, RELATIVE TO THE REPO ROOT */
"backward_exercises" : [ "exercise1", "exercise2", ... ] }
<part> = {
"subtitle": <string>,
/* Path to the exo, RELATIVE TO THE SUBINDEX */
"subexercise": <directory>,
"student_hidden": <boolean>, /* Optional: false by default */
"student_weight": <integer>,
"teacher_weight": <integer> }
{ "learnocaml_version": <string>,
"meta": <meta>,
/* Path to the ref. grader exo for retrograding, RELATIVE TO THE SUBINDEX */
"check_all_against": <directory>, /* Optional: no retrograding by default */
/* Last but not least: */
"parts": [ ( <part> )+ ] }
subindex.json
w.r.t. the use case of #395:{ "learnocaml_version": "1",
"meta": {
"learnocaml_version": "2",
"kind": "problem",
"stars": 4,
"title": "PFITA - DM1",
"identifier": "pfita-dm1",
"authors": [["Yoan Mollet", "yoan.mollet@example.com"], ["Erik Martin-Dorel", "emd@example.com"]],
"focus": ["recursion"],
"requirements": ["lists", "pattern-matching"],
"forward_exercises": [],
"backward_exercises": [] },
"check_all_against": "dm1-prof",
"parts": [
{ "subtitle": "First tests",
"subexercise": "dm1-etu",
"student_weight": 1,
"teacher_weight": 0
},
{ "subtitle": "Grading",
"subexercise": "dm1-prof",
"student_hidden": true,
"student_weight": 0,
"teacher_weight": 1
}]
}
It would be extremely useful to be able to organize an exercise into sub-parts, perhaps similarly in layout to the tutorials where you can navigate through the sections using left and right arrows at the top of the screen.
Each sub-part would have its own prelude, prepare, template/editor, description, and solution. They could all share a toplevel (though there might be some issues to work out here wrt prelude/prepare definitions).
This would be very helpful because often later parts of an exercise will rely on successfully implementing the earlier functions. For example, part a) of an exercise will require the student to implement a function
f
, and then parts b) and c) involve making use off
. If a student is unable to implementf
, they will be harshly penalized by the autograder on parts b) and c) of the exercise as well. In contrast, when manually grading an assignment we could still grade the student on parts b) and c) independently, if they are still able to usef
properly assuming a correct implementation.This is especially a problem with exercises involving modules. In a recent exercise, we wanted students to implement a module
M
with an abstract typet
, and later wanted them to implement a functorF (M): S with type t = M.t
(whereS
was some signature) and apply it toM
resulting in a moduleM2
. We were unable to test the moduleM2
since its type involved some types defined by the student in their solution, so we were not able to provide this type in our grader code.Splitting the assignment up into several exercises, each with the previous questions implemented in the prepare.ml file, would have been a way to get around this, but we felt that doing this was a bit clunky and confusing. Being able to split up an exercise into sub-exercises which can be navigated between on a single page seems like a good solution to this.