robksawyer / brenda-web

Web interface for submitting Blender render jobs to Amazon Web Services.
15 stars 4 forks source link

Build Backend Functionality for 'Add Spot Instance Render Job:Add a Job' #3

Open robksawyer opened 9 years ago

robksawyer commented 9 years ago

The following needs to happen in the backend for the Add Job function.

  1. Take submitted .blend file or .zip, check to see which was submitted
    • .blend: Zip the file up
    • .zip, .gz: Just pass the file along
  2. Create a Brenda configuration file that brenda-work will use.
  3. Run the brenda-work command and produce a Amazon SQS Queue that contains all of the frames in project. e.g. brenda-work -T ~/Brenda/frame -s 1 -e 50 push

    ~/Brenda/frame should be the local copy of frame located in lib/task-scripts/frame. Support subframe at a later date.

  4. After the frames have been processed and the queue has been established. Run the brenda-run -N 4 -p 0.07 spot command and create the spot instance requests.

    These currently take Amazon around ~5 minutes to approve. And it's worth noting that it could fail. This may be due to price or other issues.

  5. Show the status of the job.
  6. Send an email notification when the job has been approved and has started.
  7. Figure out how to stop the instances after the jobs have been completed.

I'm sure there's a lot more that needs to happen here, but this is all I could think of at the moment.

Note on implementation