DERP - objective and premise

[clararichter]

I'm still trying to make clear what we are trying to achieve with this project, and why. In this issue I hope we can elucidate the problem we are trying solve, because it's not obvious why the current system being used for assignment management has flaws.

The current situation:

• Professors of Computer Science at Colorado College uses Canvas for course management
• While Canvas offers many functionalities that are useful also for programming courses (keeping track of grades, discussion boards, messaging, etc.), it offers no support for automated and integrated assignment assessment
• The current procedure for assignment assessment is (in broad strokes and to the best of our knowledge) as follows:
  1. Grader (which may be professor or appointed student) logs in to Canvas and navigates through the user interface to assignment page.
  2. Grader downloads, for each student, the source files to their machine
  3. They run the code, potentially against some test file, and they screen the code to judge its quality

From student's perspective:

1. Student navigates the Canvas interface to assignment page, downloads to their machine necessary starter files (this typically includes assignment description, set up code, and test code)
2. Student writes the code necessary to complete the assignment, and runs potential tests locally on their machine
3. When student is happy with the source files, they go on to assignment page and submits relevant files (the format of submitted product vary depending on professor; some require the files that were created/edited by student while others want the whole project directory as a compressed file)

The premise for DERP is that the above procedures can be made more efficient, most tangibly from the professor’s/grader’s perspective. With DERP, the grader doesn’t download files to screen or run tests against. Instead, the student runs the tests via DERP (remotely) and any submitted code have a test result associated with it. The professor, then, will have all assignment submissions available in the remote repository, where they can view the source code as well as the test results of any given submission. This means that there will be no need to for a grader to download any files and run them locally.

Concern: For many assignments, there are no tests. Instead, the professor/grader compiles the code on their machine and runs the executable. Is this functionality we want to automate as well, and DERP to support?

[nasonfish]

I think this is an awesome set of goals to be looking at. I'm sure we'll be trying to refine and reframe this goal as we keep working on the project, but seeing all of this written out is awesome-- thank you Clara!

As for the concern: I'm sure there are going to be situations in which DERP's usefulness runs out, and we won't be able to do much more for the professor other than collect the code and let them download it. However, arguably, this is still better than Canvas in this case-- people upload their code in wildly different ways to canvas, from ZIP archives, TAR archives, to uploading a bunch of files individually (I hear this one is the worst of all).

Even if we can't give them an advanced suite of tests on independent projects or things that cannot be tested in a standard way, I'd argue we still offer the experience of using a standardized git workflow, and we can provide tools for downloading all of the submissions, organized by student, in bulk.

[clararichter]

So, one of the two major objectives with our system is to support and encourage students to adopt a git workflow.

For a given assignment, the student's work necessarily happens in a git repository. To turn in their assignment, they push to remote, tag their latest commit, and push the tag to remote. Whether or not they take full advantage of git and standard practices for version control is up to them but by establishing git push + tagging as the means for submission, they must at least use basic git features to some degree.

Now the question is, do we set up a Git server (like Richard Koo did in databases), or do we use GitHub for hosting? What are the pros and cons for each? ...