search

ls1intum / Artemis

Artemis - Interactive Learning with Automated Feedback
https://docs.artemis.cit.tum.de
MIT License
513 stars 294 forks source link

Possible race condition in programming exericse #3827

Open julian-christl opened 3 years ago

julian-christl commented 3 years ago

Describe the bug

This exercise had a manual test execution after the deadline was activated. The student worked close to the deadline, one push was < 1min after the deadline which was displayed as "graded", this might be some grace period of probably one minute? the last push, the one after the previous, was within 2 mins after the deadline which was not graded. When the tests got executed, they triggered the build plan which obviously checked out the latest commit. Therefore the test ran on the latest instead of the latest graded commit.

For displaying the information to the student, however, the latest graded commit gets taken which doesn't have any results because no tests were executed for it.

To Reproduce

  1. Create programming exercise with tests executed manually
  2. Submit and wait for the deadline
  3. At least 2 minutes after the deadline push again (admin permissions or give your account the writing permissions that were revoked)
  4. Check that the commit before the deadline is marked graded and the one after the deadline is marked as not graded
  5. Run the tests
  6. See that the tests run on the latest commit and not the latest graded one
  7. Check the submission from student view and see that the tests are not displayed because the latest graded commit is taken for that.

Expected behavior

Either make sure that the grace period ends after every repository was locked (preferred) => double-check or something. Or try to tell bamboo which commit to use for the build

Screenshots

No response

What browsers are you seeing the problem on?

Chrome

Additional context

My solution was to remove the commit (had to temporarily remove the "rewrite history"-protection) and rerun the test.

Apparently, it also worked when removing the commit from Artemis and retriggering the build. Somehow the commit is then reinserted but now graded. However, this is not ideal because then the student gets an unfair advantage over the others because a usually not graded commit counts.

This issue occurred to me the first time in 1.5 years so I assume this occurs only very rarely. It needs fixing anyway, though.

Relevant log output

No response

Strohgelaender commented 3 years ago

I'd prefer a solution that tells the build system which commit it should use when building the submission. A "Rebuild last graded commit"-function could also be helpful when you allow additional non-graded submissions after the due date.