Offline grading infrastructure

hazelgrove / hazel

Hazel, a live functional programming environment with typed holes

http://hazel.org/

MIT License

801 stars 52 forks source link

Offline grading infrastructure #630

Open cyrus- opened 2 years ago

LighghtEeloo commented 2 years ago

Hi @cyrus- ,

We've found a way to move Printer into core https://github.com/hazelgrove/hazel/tree/haz3l-serde-in-core . When do you have time to meet so that we can discuss on the required information to display on gradescope?

cyrus- commented 2 years ago

Required information is, for each exercise:

exercise name and number for reference
the student's test cases
the student's implementation
a grade report (text format) showing the autograder results, should look like what the UI reports, and the overall grade
the overall grade should be the point value that gradescope shows initially, and then graders can go in and do deductions from there (similar to how we did with learn ocaml)

This will require probably moving a few more things into core, e.g. stuff from SchoolExercise and Grading.

LighghtEeloo commented 2 years ago

I’m not sure, but SchoolExercise and Grading seem to be unrelated to core. There may be a different approach where we make a new lib school that doesn’t depend on web stuff and an executable that depends on school to perform serde. Maybe we can experiment on that.

cyrus- commented 2 years ago

Yeah that's fine, you can call it haz3lschool for consistency.

LighghtEeloo commented 2 years ago

I'm curious about

this. Can someone give me a clue on what's the prompt doing here?

cyrus- commented 2 years ago

The prompt is the exercise instructions. The annotations have to do with deriving. [@opaque] means that sexp deriving will not attempt to get an sexp from the Node.t, since nodes are not serializable that way. The [@printer] is for deriving show -- I'm using it for instructor mode exercise module export where the prompt is defined in a separate file, so the printer just inserts a variable "prompt" that gets bound by module export.

cyrus- commented 2 years ago

For moving it into core, you can try to make 'node parameter to p that we only instantiate in web?

LighghtEeloo commented 2 years ago

That's the approach I did; but I'm a bit concerned about the future extensibility...

LighghtEeloo commented 2 years ago

I've just pushed the attempt using this approach (01e4697) /^o^/ Want some input on how it's doing...

cyrus- commented 2 years ago

Looks likes its progressing reasonably, see comments -- I think we just want to package all school related things into a haz3lschool module.

Can you make a PR for future discussion.

LighghtEeloo commented 2 years ago

Sure! Will do once I get off the bus

LighghtEeloo commented 2 years ago

https://github.com/hazelgrove/hazel/pull/858