ese-msc / introduction-to-python

"Introduction to Python" course for future Imperial College London MSc students
BSD 3-Clause "New" or "Revised" License
75 stars 94 forks source link

Exercise 2.2: Make a table (a list of lists) of function values - Floating point PyBryt tolerance may not be sufficient #79

Closed acse-jrn22 closed 1 year ago

acse-jrn22 commented 1 year ago

In the exercise 2.2, I'm trying to use the following solution:

import numbers
import numpy as np
import pybryt
from lecture import pybryt_reference

v0 = 1
n = 11
g = 9.81 
dt = 2. * v0 / (n - 1) / g # `dt` value is    0.020387359836901122

# Uncomment the following line so that tests can pass
# dt = 2. * v0 / g / (n - 1) # `dt` value is 0.02038735983690112

tlist = []
ylist = []

for i in range(n):
    t = dt * i
    y = v0 * t - g / 2 * t**2
    tlist.append(t)
    ylist.append(y)

displacement = [tlist, ylist]

# Begin assertion checks
assert isinstance(tlist, list)
assert isinstance(ylist, list)
assert isinstance(displacement, list)
assert np.isclose(tlist[0], 0)
assert np.isclose(tlist[-1], 0.2038735983690112)
assert np.isclose(ylist[0], 0)
assert np.isclose(ylist[-1], 0)
assert len(tlist) == len(ylist) == 11
assert all([isinstance(i, numbers.Real) for i in tlist])
assert all([isinstance(i, numbers.Real) for i in ylist])
assert all([isinstance(i, list) for i in displacement])
assert all([isinstance(i, numbers.Real) for i in displacement[0]])
assert all([isinstance(i, numbers.Real) for i in displacement[0]])
assert np.allclose(displacement[0], tlist)
assert np.allclose(displacement[1], ylist)

# Begin PyBryt Checks
with pybryt.check(pybryt_reference(2, 2)):
    tlist, ylist, displacement

However, I'm getting the following PyBryt errors:

REFERENCE: exercise-2_2
SATISFIED: False
MESSAGES:
  - SUCCESS: You generated tlist correctly. Great!
  - ERROR: Your ylist is wrong.
  - ERROR: Hmm... It seems your nested displacement list is wrong.

Changing the expression dt = 2. * v0 / (n - 1) / g to dt = 2. * v0 / g / (n - 1) solves the issue, though. This makes me consider a possible insufficient error tolerance value for the PyBryt checks.

marijanbeg commented 1 year ago

Hi @acse-5d697a4f, thank you for raising this issue. We have investigated this issue and the error was raised due to the tolerances.

PyBryt was expecting a list whose elements are within 1e-5 relative tolerance from the reference solution, which works for most values in the list. However, the last element in ylist produced by your code was "nearly zero" -2.7755575615628914e-17 which is not within 1e-5 relative tolerance from 0. Therefore, we now introduced absolute tolerance 1e-8 to accommodate this case.

Thank you again for raising this issue.