Make MPT repo public again

[sapi]

@athleticus @pjritee

The repo seems to have become private over the last week or so for some reason.

I don't see any reason to keep it this way. Students can read the source code off their own computers; there's no reason to hide it from the internet. This includes the problem sets; we decided not to keep them secret.

I'd like committed students to be able to raise issues against the repo, and possibly even make contributions.

[ironstrider]

I thought we had decided to make the solutions private, and since the problem set is included, the repo should be too, therefore.

But I agree, it would be good to have the repo minus the problem set open source for students to raise issues on, fork, others to write their own problem sets, etc.

It is really @pjritee's call on this one. Personally I reckon we should split the problems from the main software and make MPT repo public and the problems/solutions repo private.

Sent from my iPhone

Ben Martin ben@benmartinstudios.com.au

On 16 Feb 2015, at 5:41 pm, Sean Purdon notifications@github.com wrote:

@athleticus @pjritee

The repo seems to have become private over the last week or so for some reason.

I don't see any reason to keep it this way. Students can read the source code off their own computers; there's no reason to hide it from the internet. This includes the problem sets; we decided not to keep them secret.

I'd like committed students to be able to raise issues against the repo, and possibly even make contributions.

— Reply to this email directly or view it on GitHub.

[jgat]

I don't envy anyone who gets given the task of cherry picking out any commits which touch the problem set (and no, simply initialising a brand new repo and throwing out the entire commit history isn't the best solution either). Apart from that, it would be straightforward on the technical level to separate the system from the problem set.

I'm also under the impression that the solutions are not included anywhere in this repo, and also that the problem set data which is in this repo is also included in the installs we give to students.

If you want a private solutions repo, go for it.

If you do end up separating the two sections, it would be courteous to create another demo problem set for inclusion in the public repo as an example, so that others looking at the software have something concrete to work with.

My only concern (at least, which comes to mind) that would cause me to side with Ben's remarks is that the testing framework is deterministic and exploitable for most (if not all) questions. Adding some nondeterminism (aka "import random") to the "alternate tests" would ease this, but only if it could be done in such a way that the solution wasn't embedded in the test code.

On Mon, 16 Feb 2015 22:04 Ben Martin notifications@github.com wrote:

I thought we had decided to make the solutions private, and since the problem set is included, the repo should be too, therefore.

But I agree, it would be good to have the repo minus the problem set open source for students to raise issues on, fork, others to write their own problem sets, etc.

It is really @pjritee's call on this one. Personally I reckon we should split the problems from the main software and make MPT repo public and the problems/solutions repo private.

Sent from my iPhone

Ben Martin ben@benmartinstudios.com.au

On 16 Feb 2015, at 5:41 pm, Sean Purdon notifications@github.com wrote:

@athleticus @pjritee

The repo seems to have become private over the last week or so for some reason.

I don't see any reason to keep it this way. Students can read the source code off their own computers; there's no reason to hide it from the internet. This includes the problem sets; we decided not to keep them secret.

I'd like committed students to be able to raise issues against the repo, and possibly even make contributions.

— Reply to this email directly or view it on GitHub.

— Reply to this email directly or view it on GitHub https://github.com/CSSE1001/MyPyTutor/issues/95#issuecomment-74491919.

[sapi]

The problems aren't encrypted anymore (they're not even compiled). There's not really any benefit in separating things when the problems are accessible on the student's computer (as they have to be, in order to be read).

The solutions aren't really accessible. There's test cases, but reading and understanding them would be more difficult than just solving the problems directly, I'd say.

At the end of the day, someone with access to the MPT source alone can just comment out the check for correctness. Short of server-side checking, we can't stop people from cheating on this (although they'll go ahead and fail everything else). I don't see why anyone would cheat by reading solutions to the tests when they could just bypass the testing stage entirely.

I was chatting with Peter the other day, and he seemed to agree that public was okay for everything.

[jgat]

Also noteworthy: if the repo is public, so are the issues and wiki. If there is any private information in there, they should be edited out (e.g. there may be issues, open or closed, which contain complete solutions to a problem).

Github issues don't show a history of edits, so any sensitive content there can safely be edited out.

On Mon, 16 Feb 2015 22:19 Jackson Gatenby jackson.gatenby@gmail.com wrote:

I don't envy anyone who gets given the task of cherry picking out any commits which touch the problem set (and no, simply initialising a brand new repo and throwing out the entire commit history isn't the best solution either). Apart from that, it would be straightforward on the technical level to separate the system from the problem set.

I'm also under the impression that the solutions are not included anywhere in this repo, and also that the problem set data which is in this repo is also included in the installs we give to students.

If you want a private solutions repo, go for it.

If you do end up separating the two sections, it would be courteous to create another demo problem set for inclusion in the public repo as an example, so that others looking at the software have something concrete to work with.

My only concern (at least, which comes to mind) that would cause me to side with Ben's remarks is that the testing framework is deterministic and exploitable for most (if not all) questions. Adding some nondeterminism (aka "import random") to the "alternate tests" would ease this, but only if it could be done in such a way that the solution wasn't embedded in the test code.

On Mon, 16 Feb 2015 22:04 Ben Martin notifications@github.com wrote:

I thought we had decided to make the solutions private, and since the problem set is included, the repo should be too, therefore.

But I agree, it would be good to have the repo minus the problem set open source for students to raise issues on, fork, others to write their own problem sets, etc.

It is really @pjritee's call on this one. Personally I reckon we should split the problems from the main software and make MPT repo public and the problems/solutions repo private.

Sent from my iPhone

Ben Martin ben@benmartinstudios.com.au

On 16 Feb 2015, at 5:41 pm, Sean Purdon notifications@github.com wrote:

@athleticus @pjritee

The repo seems to have become private over the last week or so for some reason.

I don't see any reason to keep it this way. Students can read the source code off their own computers; there's no reason to hide it from the internet. This includes the problem sets; we decided not to keep them secret.

I'd like committed students to be able to raise issues against the repo, and possibly even make contributions.

— Reply to this email directly or view it on GitHub.

— Reply to this email directly or view it on GitHub https://github.com/CSSE1001/MyPyTutor/issues/95#issuecomment-74491919.

[jgat]

It's unusually reassuring that all of the possible security issues all disappear when placed against the huge insurmountable issue that the testing step can be bypassed entirely. :/

[sapi]

It does put things in perspective.

I did look into using server-side testing, but I wasn't able to find a build of Python which was safe enough to use to exec arbitrary code on the server. The problem with a dynamic language is that it's possible to do a lot of damage in a great number of ways. Anything which locks things down sufficiently to prevent that would also cripple the language.

If this becomes a serious issue, we could always run each student's submissions in a private VM (for that student), so that they could only trash their own environment. I can't see it coming to that for the vast majority of students, though.

[pjritee]

I agree - making everything public is fine by me. As we keep saying - there are lots of ways to cheat. If students want to cheat then they may very fail the exam. The purpose of MPT is to help students and if they don't to take to opportunity to have some practice then there is nothing we can do.

[ironstrider]

Sounds like a consensus to me.