search

CodeGrade / orca

Grading job server for CS student submissions.
1 stars 1 forks source link

Orchestrator grader images not updated if Orca's base image changes. #80

Open williams-jack opened 3 months ago

williams-jack commented 3 months ago

The SHA sum of a given grader image's Dockerfile is the basis for how unique names are assigned to those images once generated by the Orchestrator's Build Service.

Unfortunately, this means that if the base image changes for any reason, then there is currently no way to force all images on the server to change -- or worse, rebuilding the image will not work as the Orchestrator will not recognize there has been an underlying change to the image due to the base image being updated.

We need to rethink how this works, with a couple ideas in mind:

  1. Should we create some sort of "rebuild all images on the server" workflow?
  2. Should we re-design how we assign a unique name to the images, where this method is able to recognize when the base image has changed?
blerner commented 3 months ago

This is trivial: the base grading image can only change when we git pull on the orchestrator, and restart the service. So, upon boot up of the orchestrator, take the SHA of the base image and compare it to the contents of some file ./last-known-base. If they differ, delete all images/*.tgz files, and update the last-known-base file contents to the new sha value. Add that file to .gitignore.

On Sun, Aug 25, 2024, 4:04 PM Jackson Williams @.***> wrote:

The SHA sum of a given grader image's Dockerfile is the basis for how unique names are assigned to those images once generated by the Orchestrator's Build Service.

Unfortunately, this means that if the base image changes for any reason, then there is currently no way to force all images on the server to change -- or worse, rebuilding the image will not work as the Orchestrator will not recognize there has been an underlying change to the image due to the base image being updated.

We need to rethink how this works, with a couple ideas in mind:

  1. Should we create some sort of "rebuild all images on the server" workflow?
  2. Should we re-design how we assign a unique name to the images, where this method is able to recognize when the base image has changed?

— Reply to this email directly, view it on GitHub https://github.com/CodeGrade/orca/issues/80, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAHAHQBICTTHYRGCFCANHVDZTI2GPAVCNFSM6AAAAABNC2C5ACVHI2DSMVQWIX3LMV43ASLTON2WKOZSGQ4DKNBUGI2TENA . You are receiving this because you are subscribed to this thread.Message ID: @.***>

blerner commented 3 months ago

Not quite as trivial as I thought: the base image also depends on the contents of the worker/ directory, since that gets copied into the container...