Mayank447 / Pseudo-Spectral-Discretization

GNU General Public License v3.0
0 stars 1 forks source link

Add tests to test for the correctness of reported eigenvalues #54

Open Mayank447 opened 1 week ago

Mayank447 commented 1 week ago

Check that the reported eigenvalues are correct (must adjust for the boundary condition) for the given spectrum.

chillenzer commented 6 days ago

I agree that it is something we could potentially test. However, I disagree that it is something that is straightforward to test because it is not an invariant of the functionality of the code but the correctness of constant data we've embedded into our code. As such, there are three options to "test" this:

  1. Spell out the expected eigenvalues by hand for a small number of test cases. This is robust in the sense that this is an invariant of the code ("a derivative operator on this lattice has these eigenvalues"). But it is error-prone (due to typos), doesn't provide great coverage (because we can't spell out a lot of those by hand), and is a nightmare to maintain (because any ever so tiny detail changed -- including floating-point rounding -- might change the numerical values and would lead to a huge update of that test data).
  2. Use the general formula for the tests (something along the lines of expected = np.fft.fftfreq(...)). This would scale well to different parameter settings. But other than that it is useless at best, if not already harmful, because it would result in copy-pasting the functional code into the tests (because that is the only fully general formula). As such, you could and likely would to exactly the same mistakes as in the functional code, so it is no gain. And furthermore, if we ever want to change the functional code internally, we'd have to remember to also change the test code to reflect that, so it might even be a harmful overspecification that could hinder development.
  3. Distill a high-level functional invariant from those eigenvalues that can be tested as a desired behaviour of the system without ever specifying the details of the data under test. This means to answer the question of "What behaviour does this data add to the code?" The answer is: It specifies which differential operator we're talking about. Thus, we could test that providing a discretised function with known derivative, we get the derivative by applying this spectrum's DiracOperator. This is basically what the examples do and it is distinct from the basic tests that we already have for the DiracOperator because we're not checking that it multiplies by some eigenvalues but that it actually produces the first-order derivative.

If anything, I'd go with the last suggestion and add an end-to-end test in the style of the examples. I've tried out the other two in a few other projects and it was never even remotely maintainable. So, to conclude: You are suggesting to test "data" as opposed to code. In order to do so meaningfully, we should test for the functionality that this data brings to the code and NOT for some fragile numerical values.