datacamp / pythonwhat

Verify Python code submissions and auto-generate meaningful feedback messages.
http://pythonwhat.readthedocs.io/
GNU Affero General Public License v3.0
69 stars 31 forks source link

Nested functions issue II: BLOCK BETA 9/15 #84

Closed hugobowne closed 8 years ago

hugobowne commented 8 years ago

In a nested function SCT, my outer function test_function_definition() does NOT like the results arg that I'm passing it:

test_function_definition("echo", body=inner_test, results=[(2)])

Maybe I am missing something about results as I raised another issue recently concerning it #83

It's exercise beginning Ch2 line341 here

filipsch commented 8 years ago

The (2) will not be a tuple. You can hack it by doing (2,), or by simply passing a list of lsits to results, it's possible since the new architecture of pythonwhat is live:

test_function_definition("echo", body=inner_test, results=[[2]])
hugobowne commented 8 years ago

great, works! another issue results though ;-) see here:

screenshot 2016-09-12 20 51 06

happy to raise another issue but thought you may have a quick suggestion?

I appreciate that the new issue is unrelated except that they're close to one another in the SCT

filipsch commented 8 years ago

Sidenote: I noticed that you had typos in the SCT https://www.dropbox.com/s/tfrqfsltl7qmts8/Screenshot%202016-09-12%2013.12.28.png?dl=0

filipsch commented 8 years ago

You can't use test_function_definition() with the results argument to see if they correctly generate a function. You cannot check the absolute equality of functions.

This would make everything very 'meta': testing a function definition to see if it generated a function, to again test whether that function is correct, would take us too far.

I would not use results. But use an 'inner_test' to check the correctness of the definition. You can also check whether the objects twice and thrice are correct.

I don't think fixing this is worth it. Hopefully, with the granular system, it will become possible to use results here, or something equivalent.

hugobowne commented 8 years ago

Right. I used results because I was stress testing the SCT wrt to returning the incorrect object, I.e. not inner_echo. I want to make sure that they return the correct object that is

filipsch commented 8 years ago

You can do that indirectly by checking the result of calling the function they just generated. You can also do

test_expression_result(expr_code = "echo(2)('test')")

Or something. This generates an object, that you can check.

hugobowne commented 8 years ago

ok cool i'm nearly there. I have taken your 2nd suggestion and when I stress test the SCT to return something else, I get the following, which is not helpful feedback for the student:

screenshot 2016-09-13 16 47 29

code:

*\ =pre_exercise_code

# pec

*\ =sample_code

# Define echo
def echo(n):
    """Return the inner_echo function."""

    # Define inner_echo
    def ____(____):
        """Concatenate n copies of word1."""
        echo_word = word1 * n
        return echo_word

    # Return inner_echo

# Call echo: twice

# Call echo: thrice

# Call twice() and thrice() then print
print(twice('hello'), thrice('hello'))

*\ =solution

# Define echo
def echo(n):
    """Return the inner_echo function."""

    # Define inner_echo
    def inner_echo(word1):
        """Concatenate n copies of word1."""
        echo_word = word1 * n
        return echo_word

    # Return inner_echo
    return inner_echo

# Call echo: twice
twice = echo(2)

# Call echo: thrice
thrice = echo(3)

# Call twice() and thrice() then print
print(twice('hello'), thrice('hello'))

*\ =sct

# All functions used here are defined in the pythonwhat Python package.
# Documentation can also be found at github.com/datacamp/pythonwhat/wiki

def inner_test():
    def inner_inner_test():
         test_object_after_expression("echo_word", extra_env = {'n' : 1}, context_vals = ['anything'])
         test_student_typed("return\s+echo_word", not_typed_msg="Did you return `echo_word`?")
    test_function_definition("inner_echo", body=inner_inner_test)

# Test: echo() definition
#test_function_definition("echo", body=inner_test)
test_function_definition("echo", body=inner_test)

test_expression_result(expr_code = "echo(2)('test')", incorrect_msg= 'When writing the function `echo`, did you return `inner_echo`?')

# Test: echo() call 1
test_function_v2(
    "echo",
    index=1,
    params=["n"]
)

# Test: twice object
test_object(
    "twice",
    do_eval=False
)

# Test: echo() call 2
test_function_v2(
    "echo",
    index=2,
    params=["n"]
)

# Test: thrice object
test_object(
    "thrice",
    do_eval=False
)

# Test: print() call
test_function(
    "print",
    not_called_msg="You don't have to change any of the predefined code.",
    incorrect_msg="You don't have to change any of the predefined code."
)

success_msg("Great work!")
filipsch commented 8 years ago

I've updated test_expression_result so you can specify an error_msg to override the default error message. Will be live together with the rest.

filipsch commented 8 years ago

As an example, here's a copy from a test I wrote:

self.data = {
    "DC_PEC": "",
    "DC_SOLUTION": "a = 2",
    "DC_SCT": "test_expression_result(expr_code = 'a', error_msg = 'cough')"
}