datacamp / pythonwhat

Verify Python code submissions and auto-generate meaningful feedback messages.
http://pythonwhat.readthedocs.io/
GNU Affero General Public License v3.0
69 stars 31 forks source link

Floating point issues when testing equality of numpy arrays #206

Closed yashasroy closed 7 years ago

yashasroy commented 7 years ago

Consider this exercise. There are multiple ways to create translated_force_b. However, as shown below, numpy does not consider a and b to be equal (seemingly because of floating point precision issues). As a result, test_object() and has_equal_ast() would not accept a solution with b, if the solution code had a.

This order of operations should not matter, and students should be able to create the array in both ways. Something like numpy.testing.assert_almost_equal() may be needed to test these kinds of cases.

screenshot 2017-04-20 17 40 08 1

cc @hugobowne

machow commented 7 years ago

where a is the object you want to test

Ex().has_equal_value(code = "np.allclose(a)")

edit: wait, I see the issue now. allclose requires both results, which is probably why Hugo suggested rounding. I tested the line below a bit, and it seems like it should work for what you're doing...

Ex().has_equal_value(expr_code = "np.round(a, decimals = 5)")
yashasroy commented 7 years ago

Thanks! This successfully resolves the issue. However, it's also an ideal example of the need for improved default messaging for has_equal_value(), raised in https://github.com/datacamp/pythonwhat/issues/208.

Without specifying a custom feedback message using the incorrect_msg parameter, this is the default error message, which is intimidating and unhelpful: screenshot 2017-04-25 12 40 40

As far as this particular issue goes, I'm fine with this solution. Since it's not something that comes up often, it's lower in priority compared to the other issues. That said, it's still worth adding this approach to the documentation.

machow commented 7 years ago

@yrdatacamp added to pythonwhat course tutorial docs float exercise

relevant piece below...

# NOTE on floating point comparisons booby traps ---------------------------
# floating point comparisons can return false, even when two sets of
# operations are mathematically equivalent. For example, x and y are
# not equal in Python
#
#   a = [1.1, 1.2, 1.3]
#   x = (sum(a[:-1])/3 + a[-1]/3)     # 3.5999999999
#   y = (a[0]/3 + sum(a[1:])/3)       # 3.6
#   x == y                            # False
#   round(x, 10) == round(y, 10)      # True
#
# In order to flexibly test for equivalence in these cases, you can
# round the results to a desired decimal place.
# For example, to test a floating point variable x,
#   Ex().has_equal_value(expr_code = 'round(x, 10)')
machow commented 7 years ago

closing, but have opened an issue for adding more flexibility to equality checks #218