search

jcrodriguez-dis / moodle-mod_vpl

Virtual Programming Lab for Moodle (Module)
GNU General Public License v3.0
103 stars 89 forks source link

Return error when evaluating a submission that contains a file with the same name as an execution file #158

Open FeldrinH opened 1 year ago

FeldrinH commented 1 year ago

As far as I understand the current behavior of VPL is that if the submission contains a file with the same name as an execution file (a file added by a teacher under 'Execution files'), then VPL will ignore the file in the submission and always use the execution file.

This behavior is safe, in that it prevents students from replacing test files and thus breaking the testing system, but it generally leads to a few unexpected behaviors that can result in strange and hard to debug problems:

  1. Part of the student's submission is ignored for no apparent reason (from the student's point of view).
  2. If anything uses VPL_SUBFILES or other related environment variables to automatically process student submitted files, then that processing will now unknowingly include some execution files that had the same name as student submitted files.

Because of these issues I propose that a better behavior would be this: If an attempt is made to evaluate a submission that includes files with the same names as files under execution files, then VPL will return an error explaining what happened to the user and refuse to evaluate the submission. This would elliminate all the aforementioned problematic behaviors.

nheir commented 6 months ago

Another way could be to place submission files into a specific directory, say (vpl_)submission instead of the user home for both execution files and student files.

FeldrinH commented 1 week ago

Another way could be to place submission files into a specific directory, say (vpl_)submission instead of the user home for both execution files and student files.

Yes. That would be a cleaner approach. Unfortunately it would also be a breaking change. Many custom scripts that teachers use for evaluation would break.

(I don't personally mind breaking changes, but I suspect that there is a sizable number of VPL users that wrote custom evaluation scripts years ago and are not too keen to dig into their old code to update them.)