datacamp / pythonwhat

Verify Python code submissions and auto-generate meaningful feedback messages.
http://pythonwhat.readthedocs.io/
GNU Affero General Public License v3.0
67 stars 31 forks source link

Backend error for certain submission #169

Closed filipsch closed 7 years ago

filipsch commented 7 years ago

Noticed a backend error on a python exercise. When I remove the SCT, there's no problem.

If you submit the following code in this exercise:

import pickle
with open('data.pkl', ('rb')) as file:
    d = pickle.load(file)    
print(file)
print(type(d))

The error is:

DataCamp encountered the following error: unorderable types: NoneType() > int()

For full source code, you can check here.

@machow can you look into this?

machow commented 7 years ago

@filip, I'm pushing a fix now. Unfortunately, this is basically a regression from when I switched to testing for PickleError, when getting representations out of processes.

machow commented 7 years ago

(the reason is that not only is a file object undillable, it causes an error if you try to dill it)

filipsch commented 7 years ago

The fix will go live in a moment, but now, for that same exercise, there is a pretty weird feedback message. This is probably due to the internal machinery of context managers.

If you submit:

import pickle
with open('data.pkl', ('rb')) as file:
    d = pickle.load(file)    
print(file)
print(type(d))

The error is "Did you call print() with the correct arguments? The first argument is incorrect" and the print call inside the with statement is highlighted...

machow commented 7 years ago

This is pretty serious. I'll fix the highlighting tonight--when I tried removing highlighting in test_function, it looks like the reporter fell back on highlighting a previous entry. The culprit line is here. In general, it's a bit harder to remove highlighting, because of the fallback behavior, and will probably need a dummy NoHighlight class to really nail down. A temporary fix would be to turn highlighting on by default in test_function.

machow commented 7 years ago

I've thought about it more, and I can easily fix this by disable highlighting completely for test_function, but when it's used as a sub-test, we'd ideally like for the highlighted portion of the outer test to show up.

However, the bigger problem is that a singleton Reporter object is used through the whole library, which leads to the risk of highlighting, etc.. "bleeding over" from one set of tests to another unrelated one. I'm going to submit a pull request tonight to re-enable function highlighting for now, and open an issue for fixing.

machow commented 7 years ago

closing, since reporter issue is opened at #172 and test_function highlighting is covered at #152