openwebwork / webwork2

Course management front end for WeBWorK
http://webwork.maa.org/wiki/Main_Page
Other
143 stars 165 forks source link

Some observations about the "Check Test" button #2287

Open Alex-Jordan opened 8 months ago

Alex-Jordan commented 8 months ago

Consider a test with multiple pages, say 3 questions per page. And it might make the following more relevant if you imagine a student typed in answers everywhere, and the answers were recorded but never scored (as apparently can happen with "start and grade" proctoring if the student never authenticated at the end).

Imagine you are acting as the student, viewing a student's quiz version. You see 0's everywhere in the table across the top, but you also see the answers they had typed in. Assume several of those answers are correct. Now click "Check Test". What happens is only the page you are on gets checked, and the table updates scores, but only for that one page. This caused confusion to an instructor for two reasons:

I'm not sure the behavior of anything needs to change, but we could do better to make clear what is going on. For "Check Test", either it could change behavior and check the whole test, or it could say something like "Check this Page". Also up in the table, we could communicate better that the scores you see are more about what is currently on the page and was checked most recently, not what the saved scores for the student are.

drgrice1 commented 8 months ago

This is not about communicating what "Check Test" does, but rather fixing a bug. Normally, when "Check Test" is clicked, all pages are scored, not just the page you are on. So something is wrong in this case. It seems there are some other things that are out of the ordinary in what you describe also.

Alex-Jordan commented 8 months ago

I'll return to the course in question and try to reproduce exactly what happened. It does include a test that was originally "start and grade" proctoring, and then after some students started, the instructor changed the template set to "start" proctoring. That caused problems that I also want to investigate, but I thought the things I mentioned here were likely independent. But maybe not. Anyway I saw these things while trying to help the instructor recover grades for the students who had started the test with "start and grade" proctoring still in effect.

somiaj commented 8 months ago

@Alex-Jordan that seems similar to my experience of how checking tests work after they are due. Note when you hit check test, there are messages that state that the grade is not recorded (as shown in the screenshot below).

image

I do agree with you that updating the % score table is probably not ideal in this case, but since I have learned that is how it works, I've just ignored that.

I can't state anything about checking test only checking one page at a time, since when I use this I'm only looking at one page at a time and students grades on other pages aren't changing due to saved answers like the case you ran into. The grades are cached in hidden fields, so I'm not surprised there. Might be the code has assumed that since the test is already graded that only the current page needs to be checked and then uses the cached values for all other pages.

drgrice1 commented 7 months ago

I realized that I was mistaken in my earlier comment that when "Check Test" is clicked all problems are scored. Only the problems on the page are processed in that case. The only time that all problems are processed even if they are not on the page is when "Grade Test" is clicked.

One way to resolve the issues here would be to change that so that "Check Test" also processes all problems. Although that adds to the server load. Then there will be more of the heavy hits that currently only occur when a test is graded.