Fix order of `#pragma repy` file name and args

[aaaaalbert]

64 noted that this test would PASS although it shouldn't:

#pragma repy restrictions.default additionalarg
# --- Note the trailing additional arg above! ---
#pragma out
raise RepyException()

while this one FAILed correctly:

#pragma repy restrictions.default
#pragma out
raise RepyException()

Inspecting the command that utf.py prepares to run the problematic test case exhibits that the additionalarg gets spliced in at the wrong place: ['/usr/local/opt/python/bin/python2.7', 'repy.py', 'restrictions.default', 'additionalarg', 'ut_problematic_test.r2py']

Thus, the sandbox is instructed to start additionalarg as the program, not the test case! This doesn't exist, so repy.py just prints FATAL ERROR: Unable to read the specified program file: 'additionalarg' to stdout, and exits. Since this example test case specifies #pragma out, Repy's error message isn't considered a FAIL.

The fix obviously is to order the args to the sandbox so that the program name goes first, etc.

[JustinCappos]

How do you handle things like security layers though? I think this behavior was intentional to handle those cases...

On Tue, Jul 19, 2016 at 4:21 PM, aaaaalbert notifications@github.com wrote:

64 https://github.com/SeattleTestbed/utf/issues/64 noted that this

test would PASS although it shouldn't:

pragma repy restrictions.default additionalarg

--- Note the trailing additional arg above! ---

pragma out

raise RepyException()

while this one FAILed correctly:

pragma repy restrictions.default

pragma out

raise RepyException()

Inspecting the command that utf.py prepares to run the problematic test case exhibits that the additionalarg gets spliced in at the wrong place: ['/usr/local/opt/python/bin/python2.7', 'repy.py', 'restrictions.default', 'additionalarg', 'ut_problematic_test.r2py']

Thus, the sandbox is instructed to start additionalarg as the program, not the test case! This doesn't exist, so repy.py just prints FATAL ERROR: Unable to read the specified program file: 'additionalarg' to stdout, and exits. Since this example test case specifies #pragma out, Repy's error message isn't considered a FAIL.

The fix obviously is to order the args to the sandbox so that the program name goes first, etc.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/SeattleTestbed/utf/issues/75, or mute the thread https://github.com/notifications/unsubscribe-auth/AA0XD2yDaoRYH4BCIapOkXi2EYwJDie6ks5qXTG9gaJpZM4JQIVd .

[aaaaalbert]

1) There's a command-line option for running tests with specific, user-supplied security layers enabled: python utf.py -s blur_location.r2py -a

2) I don't see existing test cases attempting to insert security layers before themselves; however, the current behavior is also required for dylink.r2py which we use a lot. At the same time, this behavior keeps test cases from specifying their callargs (on which production code relies frequently).

The underlying problem is that a #pragma repy is designed to fill in the details (repy.py, a default restrictions file, and the test case's file name) for you if you omit them, but then it becomes ambiguous where the security layers end and the other arguments start, i.e where on the line to insert the program's name.

Maybe a separate #pragma callargs would be the cleanest solution. It would also work for non-Repy tests, and requires no changes to existing tests that don't need callargs.