We are currently testing the correctness of the implementation differently for each neural network, depending on how easy it is to access the original implementation and what reasonable tests can be performed. In doing so we also install the reference implementations in the CI conda environment.
I think we should remove the development tests and the dependencies that the comparison with the original implementation involves. After implementing a neural network, the final test should compare the implementation to the original implementation by transferring weights and biases and ensuring that the output is the same. This seems like a large time investment but can be done in a day. This is then the only test that we keep in the CI --- we test that given the weights and biases, we get a specific output for the energies/forces and other properties of interest.
@ArnNag and @chrisiacovella , what is your opinion on this?
We are currently testing the correctness of the implementation differently for each neural network, depending on how easy it is to access the original implementation and what reasonable tests can be performed. In doing so we also install the reference implementations in the CI conda environment.
I think we should remove the development tests and the dependencies that the comparison with the original implementation involves. After implementing a neural network, the final test should compare the implementation to the original implementation by transferring weights and biases and ensuring that the output is the same. This seems like a large time investment but can be done in a day. This is then the only test that we keep in the CI --- we test that given the weights and biases, we get a specific output for the energies/forces and other properties of interest.
@ArnNag and @chrisiacovella , what is your opinion on this?