Checking script

[dluschan]

Hello! Help me please to solve a problem. I need to check student’s program which splits input text according some rules. I can’t generate all solutions of this problem because there are too many of them. But I can check output of student’s program. Now my vpl_evaluate.sh checking script looks like this:

#! /bin/bash

cat > vpl_execution <<EOF
#! /bin/bash
for i in {1..10}
do
    python3 generate.py > input.txt
    python3 student.py < input.txt > output.txt
    cat input.txt output.txt | python3 check.py >> result.txt
done
python3 result.py < result.txt
EOF

chmod +x vpl_execution

There are many weaknesses of this checking script:

1. it supports only one language (python3) of student’s program,
2. It requires specific filename of student’s program,
3. debug and run modes need similar scripts, etc. I am sure that there is easier solution of my problem. Could you help me to find it.

[jcrodriguez-dis]

Hello Dmitry, you can use the default vpl_run.sh. VPL choices compiler based on the student's files extension. This will avoid the use of file names or compiler specific. Your code may look like this

#!/bin/bash
. vpl_run.sh
if [ -e vpl_execution ]  ;  then
     mv vpl_execution student_program
else
     exit 1
fi
cat > vpl_execution <<EOF
#! /bin/bash
for i in {1..10}
do
    python3 generate.py > input.txt
    student_program < input.txt > output.txt
    cat input.txt output.txt | python3 check.py >> result.txt
done
python3 result.py < result.txt
EOF

chmod +x vpl_execution

[dluschan]

Thanks a lot! I managed to check the works of students in different programming languages. But I had to write code for parsing cases and evaluation of solutions. Here's what I got: vpl_evaluate.sh

#!/bin/bash
. vpl_run.sh
if [ -e vpl_execution ]  ;  then
     mv vpl_execution student_program
else
     exit 1
fi

python3 read.py
declare -i k=0
echo "#! /bin/bash" > vpl_execution
for test in *.test
do
    k=$k+1
    echo "echo $test >> result.txt" >> vpl_execution
    echo "./student_program < $test > output.txt" >> vpl_execution
    echo "cat $test output.txt | python3 check.py >> result.txt" >> vpl_execution
done
echo "python3 result.py $k < result.txt" >> vpl_execution

chmod +x vpl_execution

result.py

import sys

def comment(s):
    '''formats strings to create VPL comments'''
    print('Comment :=>> ' + s)

def grade(num):
    '''formats a number to create a VPL grade'''
    print('Grade :=>> ' + str(num))
r
res = 0
n = int(sys.argv[1])
for i in range(n):
    test = input().split('.')[0]
    score = float(input())
    res += score
    comment('Testing ' + str(i+1) + '/' + str(n) + ' : ' + test + (' ok' if score == 1 else ' failed'))
    if score < 1:
        comment(input())

grade(int(100 * res / n))

read.py

for test in filter(len, ''.join([line for line in open('vpl_evaluate.cases')]).split('case=')):
    name, data = test.split('input=')
    print(data.strip(), file = open(name.strip() + '.test', 'w'))

Tell me, please, is there any way to simplify the code or to use the standard functions to parse the vpl_evaluate.cases file and evaluate solutions? Thank you.

[jcrodriguez-dis]

Hello Dmitry, I think that for the type of tests you are doing with VPL 3.2.X your solution is simple. Again I think that VPL 3.3 might reduce the code you need write.

Best regards, Juan Carlos.

[dluschan]

Thank you.

[dluschan]

So, new version VPL was released. Could you tell me, plz, how solve my problem easier. Thanks.

[jcrodriguez-dis]

Hello dluschan, I understand that you want to test students programs that must read a text file a produce a desired output. VPL prior to V3.3 allow you set the standard input you send to the students programs and the expected output that, in your case, seems to be of type exact text. The new V3.3 allow you to use program arguments to set student programs command line parameters as the input file, output file etc. You can still check the student program output without need to program code. The problem here may be the size of the text to check. Other possibility is that you ask the students to write the output in a fil. In this case you must write a test program that run the student program and test the correctness of the output file. Your test program can receive the files as program arguments. The test program must report to the "VPL default test program" the result of the test by an exit code. I think that this approach help you to reduce the need of programming. If you let me know the exact nature of your activity I can help you better.

Best regards, Juan Carlos.

[jcrodriguez-dis]

Beware that the use of a program that generate the correct output to compare it with the students output is a security issue. Also is a security problem using a program to check students result if it is written in an interpreted language and the students programs can access its code.

[dluschan]

Hello jcrodriguez-dis,

now I have to write a lot of code for a simple check: http://demovpl.dis.ulpgc.es/moodle/mod/vpl/view.php?id=525 Can this be made easier?

Best regards, Dmitry Luschan.

[jcrodriguez-dis]

Hello Dmitry see this solution http://demovpl.dis.ulpgc.es/moodle/mod/vpl/view.php?id=526

Best regards, Juan Carlos.