ossu / computer-science

🎓 Path to a free self-taught education in Computer Science!
MIT License
170.07k stars 21.47k forks source link

coursera's auto-grader issues in "programming languages part A" persisting for over a year #547

Closed MRensen closed 5 years ago

MRensen commented 5 years ago

In "Programming Languages- Part A" I've come across this issue 3 times now. You upload your submission to the auto-grader, but it returns with the error "submission exceeded allotted grading time". Nothing's wrong with my submission though, it's a known issue with coursera's auto-grader that's been around for over a year (as can be seen in the discussion forum). Biggest problem is, you still have to wait 24 hours to re-submit your work.

I understand that this course is much loved amongst most of you, so throwing it out is probably not an option. But, maybe we can collectively urge coursera to adress the issue, like, write a letter or a petition or something? Any thoughts on the matter?

joshmhanson commented 5 years ago

I see what you mean from looking at the discussion forum; I found instances of that error in several other weeks. This thread is especially informative, although I have to regard the students' statements with some caution: https://www.coursera.org/learn/programming-languages/discussions/weeks/2/threads/ilujRPYqEeenDhK11AXZ-A

From what I've seen, Coursera's platform has a feature that automatically cuts short the running of a function after an allotted time to ensure their systems don't get bogged down in a single student's computations, i.e. to prevent infinite loops, memory leaks, etc. But the way they implemented this is buggy, and if their auto-grader believes that your function is running too long even when it's not, it will cut it short pre-maturely.

So there may be cases where the student has legitimately created a solution with correct asymptotic complexity and the Coursera platform is failing, but some of the complaints may be coming from students whose solutions are in fact too inefficient.

Thus whenever Coursera's systems fail, they can just lazily blame the student for writing an inefficient function even if that's not really what happened. The black box nature of Coursera's infrastructure is a problem for MOOCs; it doesn't scale (economically). To put it another way, I believe there's not enough money to be made in MOOC platforms (as a business) to justify paying people to solve little problems like this, yet such little problems can bring a whole course crashing down. I wish MOOC platforms would adapt Exercism's infrastructure.

As for what to do: before anything else, definitely make sure you have followed Sean's advise (in the linked thread) to report the issue to Coursera directly in the submission page right as the issue occurs. It clearly won't solve the issue, but if we complain to them through another means and you haven't done this step, Coursera will just whine that you didn't follow their instructions. A petition might be worth doing just to see how Coursera deals with it, but I don't have high hopes that they will act.

yoosif0 commented 5 years ago

I think this happens when you submit more than one assignment at the same time.