SeattleTestbed / utf

Unit Test Framework for SeattleTestbed
MIT License
1 stars 9 forks source link

utf doesn't correctly capture stdout in some cases... #60

Closed JustinCappos closed 8 years ago

JustinCappos commented 9 years ago

utf doesn't seem to capture stdout correctly in all cases. Consider the following test:

#pragma out Code failed safety check

# code safety error on the next line
import socket

utf will say that stdout is empty, even though if you run repy by hand on the command line (not w/ utf) it prints this message. Furthermore, if you redirect stdout from the command line run (not w/ utf), you do see the text.

aaaaalbert commented 9 years ago

Did you mean to test this:

#pragma repy
#pragma out Code failed safety check
import socket

This works correctly in utf.py (in the sense that it expects the error message that is printed on stdout):

$ python utf.py -f ut_import.r2py
Testing module: import
    Running: ut_import.r2py                                     [ PASS ]
$ python repy.py restrictions.default ut_import.r2py
Specified repy program is unsafe!
Static-code analysis failed with error: Code failed safety check! Error: ("<class 'exception_hierarchy.CheckNodeException'> (5, 'Import')",)

For the original code snippet that doesn't have #pragma repy, the import is not an error, and the code indeed doesn't print anything, so it's the #pragma out that's in error here.

$ python ut_import.py
$ 
aaaaalbert commented 8 years ago

Looks like it was the missing #pragma repy in the original issue description that caused the confusion. There doesn't seem to be an actual problem. I'm closing the issue as invalid.