Ability to specify versions of submissions/grade reports

michaelwisely commented 8 years ago

Grader should be able to handle multiple imports / multiple regrades more cleanly. Consider this situation:

You download everything from Blackboard at the submission deadline. Then, you import all submissions into Grader. A day later, you download everything from Blackboard again (because who wants to go through and just cherry pick the late submitters). It would be convenient if, when grading, you could specify to Grader that you only want to grade the first submission for each student.

In that example, we would be importing a student's submission twice if they submitted on time. But we're too lazy to filter out lates, so there they are. Duplicates.

We should use a cool syntax like Git does with rev-parse. Maybe something like...

# Grade this student's most recent submission for assignment "hw01"
grader grade hw01 fmm000

# Grade this student's second-most recent submission for "hw01"
grader grade hw01 fmm000^

# Grade this student's nth-most recent submission for "hw01"
grader grade hw01 fmm000@{n}

# Grade this student's oldest submission for "hw01"
grader grade hw01 fmm000@{-1}

# Grade all old submissions
grader grade hw01 @{-1}

We could do something similar for generating or retrieving specific reports.

brhoades commented 8 years ago

This is an excellent idea, it should be modularized too. Modules like inspect / review (on its way) would both benefit from being able to select containers this way.

michaelwisely commented 8 years ago

list needs an update as well. There needs to be a way to group submissions by student instead of just listing each submission separately.

On Mon, Jan 25, 2016 at 3:10 PM, Billy Rhoades notifications@github.com wrote:

This is an excellent idea, it should be modularized too. Modules like inspect / review (on its way) would both benefit from being able to select containers this way.

— Reply to this email directly or view it on GitHub https://github.com/grade-it/grader/issues/14#issuecomment-174664003.

tymorrow commented 8 years ago

Here are some notes from discussion that Mike and I had today.

Firstly, grader's grade subcommand would behave in a manner consistent with the output from the list command.
So in addition to using list to display any information you want to see about submissions, you could use it, in conjunction with some provided reference, to verify the submissions grade would going operate on before doing it. This will become more clear as we go on.

`list` default behavior

By default, list will output all of the submissions for every assignment it has.

$ bin/grader list
hw01
  Student 1 Submissions
    54558482 [Not graded]  ...
    49200490 [Graded] ...
      Results 84847339
      ...
    87573931 [Graded] ...
      Results 77747433
    ...
  Student 2 Submissions
    92040076 [Not graded] ...
    ...
  ...
hw02
  Student 1 Submissions
    75849393 [Not graded]  ...
    17274950 [Not graded] ...
    ...
  Student 2 Submissions
    55440222 [Not graded] ...
    ...
  ...
...

Not that it has to be this way, but in this example HW is sorted by its number, while submissions are sorted by time in descending order. Notice that by default, list shows EVERYTHING down to the grading results for each submission as a nice little directory structure.

So harkening back to when I mentioned grade will behave similarly to list, if you ran:

$ bin/grader grade

that might try to grade everything shown above. Right now it's just an error, it maybe it should stay that way. The point is that I want you to visualize list as a check for any subcommands that perform an action, such as grade or report.

So now let's talk about how list behaves in conjunction with a provided UUID or reference.

`list` with a UUID

Since every submission is associated with some globally unique id, you can simply run:

$ bin/grader list <some hash>

and it would display one record. Obviously this isn't very meaningful, BUT what if we used grade in place of list?

$ bin/grader grade <same hash>

This would translate into grader attempting to grade that single submission. However, what if that submission has already been graded? Well, then you would need to use:

$ bin/grader grade <same hash> --rebuild

which is nothing new.

`list` with a Reference

Alternatively, you can provide a reference that refers to one or more items. A reference follows the following format:

[<hw>][:<student>][@<submission>][#<result>]

Rules

All parts (things within square brackets) are optional
- Excluding one implies you want everything for that particular level
Order matters (you can't rearrange them)
If you include a symbol for some part, a subsequent value is MANDATORY
<hw> and <student> are strings that allow wildcards
<submission> and <result> are either:
- integers that count backwards from most recent
- slices using {start:end} notation

Examples

Example 1

$ bin/grader list 
$ bin/grader list *
$ bin/grader list *:*

Both produce the same result, but the last one is overly explicit and will actually perform a wildcard match operation, while a blank one will just return everything. Note that no error is thrown for the first command because <hw> does not have a symbol.

Example 2

The following commands would return the directory structure shown above but only for hw01, not all assignments.

$ bin/grader list hw01
$ bin/grader list hw01:*

While both produce the same result, the last one is explicit and will actually perform a wildcard match operation, while a blank one will just return everything.

Example 3

The following commands would generate an error based on one or more of the aforementioned rules:

$ bin/grader list @
-------------------^ Expecting value
$ bin/grader list :@0#0
-------------------^ Expecting value
$ bin/grader list *:
--------------------^ Expecting value

Again, providing a symbol means you must specify a value immediately after.

I HAVE MORE TO ADD - STANDBY.

redkyn / grader