Solution Generator: Verify via Sample Submission

[mxhdev]

Introduction

When the solution generator has finished generating a solution, the solution should be executed with the input statements in order to check, if the solution can even be solved.

Writing a Sample Submission

It might also be useful, to write a correct solution to a file as "sample submission". This file will be different from the original input file because of the following reasons:

1. Will start with defining the tags in the first line
2. Does not contain static tags.

   Problem 2 - Further Explanation

The submission reader reads sql statements per-tag. This means, that the scope of a tag is considered to start at the definition of its tag (i.e. /*1a*/) and ends as soon as a new (valid) tag or the file end is reached. This suggests that, if the original solution definition file would be used as a sample submission, the algorithm would assign multiple queries to a single tag. Because the testing process depends on the execution of only one sql statement at a time, this would break the process.

This is why it is needed to generate a separate submission file. This file does not have to be used in the verification process, but it could be useful in order to re-evaluate the solution at a later point in time.

[mxhdev]

Tag definitions are not part of the student submission. They are part of the final solution.txt file

[mxhdev]

Done since commit 7fc0ce8

Everything was tested and worked.

You can test it yourself: If the counts which are produced by the verify function have

errors + #ignored == 0 then everything is working