search

autolab / Autolab

Course management service that enables auto-graded programming assignments.
http://www.autolabproject.com/
Apache License 2.0
752 stars 214 forks source link

Job view shows me much less #324

Closed robsimmons closed 9 years ago

robsimmons commented 9 years ago

I started a bunch of regrades, but I'm unable to monitor the progress very well, because every task looks like a * to me, same as if it was a job for another course.

robsimmons commented 9 years ago

Also, how is the queue ordered? I'm not going to file a separate bug because I'm not sure what's going on, but it looks like the job number wrapped around from 500 to 1 and then the queue started working on the low-numbered assignments, stalling the high-numbered ones.

(Since the queue could concievably have more than 500 things on it there's another issue there.)

dlbucci commented 9 years ago

We'll take a look at this. The queue order is based on Tango, so @mihirpandya and @yrkumar could tell you more, but basically a 512-size array/dictionary keeps all the jobs, which explains why the wraparound occurs. It's one of the weirder code points for Tango, actually, but I think there's a reason for why it's there.

robsimmons commented 9 years ago

What would happen if I tried to regrade all for a class with 600 people?

dlbucci commented 9 years ago

So we've had decent success regrading classes of that size (typically single digit number of failures), but the success rate seems to be proportional to the upload size, so if you're moving a large files (autograder or submissions), you might have less success.

robsimmons commented 9 years ago

No, I just mean, what if there are more than 512 things on the queue? Does the world end?

droh commented 9 years ago

If there are no available slots in the queue, the job is rejected. We never anticipated such large classes at CMU. We should probably bump up the size at some point.

-- David O'Hallaron -- Prof of CS and ECE, Carnegie Mellon University -- GHC 7517, x8-8199 -- http://www.cs.cmu.edu/~droh

On Tue, Feb 24, 2015 at 12:08 PM, Robert J. Simmons < notifications@github.com> wrote:

No, I just mean, what if there are more than 512 things on the queue? Does the world end?

— Reply to this email directly or view it on GitHub https://github.com/autolab/Autolab/issues/324#issuecomment-75798351.

robsimmons commented 9 years ago

Makes sense. 122 is a year or two from there, but it seemed like it could be a problem for 213 needing to regrade all already!

On Tue, Feb 24, 2015 at 12:23 PM, droh notifications@github.com wrote:

If there are no available slots in the queue, the job is rejected. We never anticipated such large classes at CMU. We should probably bump up the size at some point.

-- David O'Hallaron -- Prof of CS and ECE, Carnegie Mellon University -- GHC 7517, x8-8199 -- http://www.cs.cmu.edu/~droh

On Tue, Feb 24, 2015 at 12:08 PM, Robert J. Simmons < notifications@github.com> wrote:

No, I just mean, what if there are more than 512 things on the queue? Does the world end?

— Reply to this email directly or view it on GitHub https://github.com/autolab/Autolab/issues/324#issuecomment-75798351.

— Reply to this email directly or view it on GitHub https://github.com/autolab/Autolab/issues/324#issuecomment-75801380.

Robert J. Simmons simrob.com rjsimmon@cs.cmu.edu robsimmons@gmail.com Cell: 404-273-6890

dlbucci commented 9 years ago

Note that as a general rule, during a regradeAll, jobs start getting finished and removed from the queue before the front-end finishes uploading files, and with enough VMs, the queue is cleared faster than the jobs are uploaded, so requests to grade more than 512 submissions (to a point) will probably be okay.

robsimmons commented 9 years ago

Fair enough: not a pressing issue.

droh commented 9 years ago

I think the solution is for the front-end to back off and retry if a job is rejected because the queue is full. I'll add this as an issue.

-- David O'Hallaron -- Prof of CS and ECE, Carnegie Mellon University -- GHC 7517, x8-8199 -- http://www.cs.cmu.edu/~droh

On Tue, Feb 24, 2015 at 1:09 PM, Robert J. Simmons <notifications@github.com

wrote:

Fair enough: not a pressing issue.

— Reply to this email directly or view it on GitHub https://github.com/autolab/Autolab/issues/324#issuecomment-75811880.

dlbucci commented 9 years ago

I'm currently adding support for the front-end to retry autograding jobs if there's no feedback or parsing fails, which should help with this. Tango would just need to send back nothing and the effect would be the same.

robsimmons commented 9 years ago

Okay. Also, is there any clue why I see "*" for everyone instead of seeing my student's jobs?

dlbucci commented 9 years ago

Yes, I will fix that right now.