ocaml-sf / learn-ocaml

A Web Application for Learning OCaml
https://ocaml-sf.org/learn-ocaml/
MIT License
303 stars 66 forks source link

Feature request: multi-part exercises #331

Open adhameer opened 5 years ago

adhameer commented 5 years ago

It would be extremely useful to be able to organize an exercise into sub-parts, perhaps similarly in layout to the tutorials where you can navigate through the sections using left and right arrows at the top of the screen.

Each sub-part would have its own prelude, prepare, template/editor, description, and solution. They could all share a toplevel (though there might be some issues to work out here wrt prelude/prepare definitions).

This would be very helpful because often later parts of an exercise will rely on successfully implementing the earlier functions. For example, part a) of an exercise will require the student to implement a function f, and then parts b) and c) involve making use of f. If a student is unable to implement f, they will be harshly penalized by the autograder on parts b) and c) of the exercise as well. In contrast, when manually grading an assignment we could still grade the student on parts b) and c) independently, if they are still able to use f properly assuming a correct implementation.

This is especially a problem with exercises involving modules. In a recent exercise, we wanted students to implement a module M with an abstract type t, and later wanted them to implement a functor F (M): S with type t = M.t (where S was some signature) and apply it to M resulting in a module M2. We were unable to test the module M2 since its type involved some types defined by the student in their solution, so we were not able to provide this type in our grader code.

Splitting the assignment up into several exercises, each with the previous questions implemented in the prepare.ml file, would have been a way to get around this, but we felt that doing this was a bit clunky and confusing. Being able to split up an exercise into sub-exercises which can be navigated between on a single page seems like a good solution to this.

erikmd commented 3 years ago

Dear @adhameer, @hannelita, @yurug, and all people interested in this feature,

FYI this use case (multi-part exercises) appears to be quite close to the use case #395 (dual-grader exercise), so @YoanwM and I are considering to devise an implementation that addresses both use cases:

So we may post in this issue (or in #395) more details in the upcoming weeks, but the overall idea is as follows:

erikmd commented 3 years ago

(As an aside, the runnning example (exercises involving modules) in @adhameer's post does not necessarily require some multi-part exercise support: it appears we can implement a single grader with independent tests, for dependent modules/functors… but indeed, it'll be much more natural to benefit from this multi-part feature to achieve this :)

erikmd commented 3 years ago

Hi @yurug @YoanwM, following our last video meeting and for the record, here is the recap of the format we finally proposed for subindex.json :relieved:

Grammar of subindex.json:

<directory> = <string>

<meta> = {
  "kind"               : ( "exercise" | "problem" | "project" ),
  "stars"              : [1 .. 5],
  "title"              : "Title of the multi-part exercise",
  /* In an exercise repository, each exercise must have a unique identifier. */
  "identifier"         : "some_unique_identifier",
  /* Authors with their emails */
  "authors"            : [["First Last", "first.last@example.com"], ...],
  /* The skills and concepts that are practiced by this exercise. */
  "focus"              : ["skill1", ..., "skillN", ..., "concept1", ..., "conceptM"],
  /* The skills and concepts that are required to do this exercise. */
  "requirements"       : ["skill1", ..., "skillN", ..., "concept1", ..., "conceptM"],
  /* The suggested exercises in case of success, RELATIVE TO THE REPO ROOT */
  "forward_exercises"  : [ "exercise1", "exercise2", ... ],
  /* The suggested exercises in case of difficulty, RELATIVE TO THE REPO ROOT */
  "backward_exercises" : [ "exercise1", "exercise2", ... ] }

<part> = {
  "subtitle": <string>,
  /* Path to the exo, RELATIVE TO THE SUBINDEX */
  "subexercise": <directory>,
  "student_hidden": <boolean>, /* Optional: false by default */
  "student_weight": <integer>,
  "teacher_weight": <integer> }

{ "learnocaml_version": <string>,
  "meta": <meta>,
  /* Path to the ref. grader exo for retrograding, RELATIVE TO THE SUBINDEX */
  "check_all_against": <directory>, /* Optional: no retrograding by default */
  /* Last but not least: */
  "parts": [ ( <part> )+ ] }

Complete minimal example of subindex.json w.r.t. the use case of #395:

{ "learnocaml_version": "1",
  "meta": {
    "learnocaml_version": "2",
    "kind": "problem",
    "stars": 4,
    "title": "PFITA - DM1",
    "identifier": "pfita-dm1",
    "authors": [["Yoan Mollet", "yoan.mollet@example.com"], ["Erik Martin-Dorel", "emd@example.com"]],
    "focus": ["recursion"],
    "requirements": ["lists", "pattern-matching"],
    "forward_exercises": [],
    "backward_exercises": [] },
  "check_all_against": "dm1-prof",
  "parts": [
  { "subtitle": "First tests",
    "subexercise": "dm1-etu",
    "student_weight": 1,
    "teacher_weight": 0
  },
  { "subtitle": "Grading",
    "subexercise": "dm1-prof",
    "student_hidden": true,
    "student_weight": 0,
    "teacher_weight": 1
  }]
}