search

qiskit-community / qiskit-dynamics

Tools for building and solving models of quantum systems in Qiskit
https://qiskit-community.github.io/qiskit-dynamics/
Apache License 2.0
105 stars 61 forks source link

Fix maxOS CI test failures #263

Closed DanPuzzuoli closed 1 year ago

DanPuzzuoli commented 1 year ago

Summary

The macOS tests in CI are currently randomly failing. So far these seem to all be using JAX, though the tests don't have randomness, so it's unclear why these failures are random.

In the past we've had issues with tests reliant on the solutions to ODEs specified with a given numerical tolerance. If the equality tolerance of the test is too close to the tolerance used to solve the ODE, a small change to the solution due to a change in the implementation of some calculation can sometimes push the test over the edge of failure (even though the solution is still fine). I'm going to try to slightly relax the equality tolerances in the offending tests and see if that helps. (This still doesn't explain the randomness, unless there are minor variations in hardware of the testing machines.)

Details and comments

DanPuzzuoli commented 1 year ago

I'm still confused as to why this was happening, but I've converted all tests in the offending file to explicitly use reasonable tolerances and the tests are now reliably passing (I've rerun them all multiple times).