search

RunestoneInteractive / RunestoneServer

Interactive books for computer science and mathematics
http://runestoneinteractive.org
Other
575 stars 503 forks source link

Autograding sometimes giving 200% as scores #1897

Closed bhoffman0 closed 2 years ago

bhoffman0 commented 2 years ago

A teacher reported that since the big server updates, she's noticed some students getting more than 100% in autograding assignments. For example, for the student with the user name minche262 in the course WHS-2021-P3. Here are some screenshots. Below this one is 5 points, but he got 9 points in autograding.

image Below this one is 5 points, but he got 10 points... image (1) And 200% in the gradebook: image (2)

I was not able to duplicate this, and I can't think what could be causing it.

bnmnetp commented 2 years ago

I'm writing this as I investigate, so excuse the stream of consciousness like narrative...

I can see that in the database there are two rows for minche262 in the question_grades table. The two rows have indexes just one apart meaning they were inserted one after the other, one has a deadline and one does not.

My first inclination is that this was some kind of database hicup. Especially since I haven't had other reports of grading irregularities, and I have not been able to reproduce it either. We use a built in function of web2py called insert_or_update to add a row to this table, so it could be a bug in the web2py database code that is causing this.

In fact it looks like I may have lost a constraint in the database migration that was ensuring this uniqueness. I need to do a little more investigating before I am confident.

bnmnetp commented 2 years ago

OK, fun fact - 0.007% of the question grades were duplicates, all recent (since the db upgrade) Thats a pretty good sign that I lost a constraint. I have:

  1. removed the duplicate entries.
  2. added a constraint to keep duplicates from happening.

Instructors will need to re-run the autograder -- just to update the totals -- for any assignment where they had this happen.

I'm pretty confident this will fix the reported issue. And the tiny percentage also explains why it has not been reported by others.

bhoffman0 commented 2 years ago

Thank you! Impressive bug finding! I'll let the teacher know.

bhoffman0 commented 2 years ago

The teacher confirmed that this fixed the problem for her assignments! Thank you!