Is your feature request related to a problem? Please describe.
At the moment, we annotate the value in the reference solution, and PyBryt checks if that value is present in the student's implementation. The way the equivalence is determined between two values (reference and student values) depends strongly on what is implemented in PyByrt (e.g. tolerances, invariants, etc.). Exposing the equivalence function to the user would allow any equivalence check, invariant, tolerance, etc. to be defined by the user in the reference solution. This way everything is exposed to the user, and the user does not rely on whether that functionality was implemented in PyBryt or not.
Describe the solution you'd like
Let us say we want to check if the student has a list in their memory footprint and we want all values to be within some absolute and/or relative tolerance.
def my_eq1(ref, value):
return np.allclose(ref, value, atol, rtol=1e-5) # here value can be any iterable
def my_eq2(ref, value):
return isinstance(value, list) and np.allclose(ref, value, atol, rtol=1e-5) # here value must be a list
def my_eq3(ref, value):
return isinstance(value, (list, tuple)) and np.allclose(ref, value, atol, rtol=1e-5) # here value can be a list or a tuple
pybryt.Value(solution, eq=my_eqX) # because equivalence is exposed, full freedom is given to the user
Is your feature request related to a problem? Please describe. At the moment, we annotate the value in the reference solution, and PyBryt checks if that value is present in the student's implementation. The way the equivalence is determined between two values (reference and student values) depends strongly on what is implemented in PyByrt (e.g. tolerances, invariants, etc.). Exposing the equivalence function to the user would allow any equivalence check, invariant, tolerance, etc. to be defined by the user in the reference solution. This way everything is exposed to the user, and the user does not rely on whether that functionality was implemented in PyBryt or not.
Describe the solution you'd like Let us say we want to check if the student has a list in their memory footprint and we want all values to be within some absolute and/or relative tolerance.