Add finite difference tests to all `operations`

[Luthaf]

This should be able to catch most mistakes regarding gradients. It would either require to add a dependency on rascaline in tests, or write a small calculator in pure Python to be used in tests. I think the second option is best and should be relatively simple.

[agoscinski]

I will make an attempt using rascalines test python/rascaline-torch/tests/autograd.py as base. Probably for the first attempt I only write the tests for torch array backend, because I am not sure how to do these tests for numpy-backend.

[Luthaf]

I don't think the autograd checks in rascaline will help here. The idea is to check explicit forward gradients stored in ˋblock.gradient(...)`, which is not integrated with torch autograd.

A better example would be in rascaline ˋtest_utilsrust module in ˋcalculators, where we do the kind of explicit finite difference that is required here.

[agoscinski]

Yes realized also that the gradcheck function is more for checking backward function implementations and cannot be reused for forward gradients (at least from what I understood). I was trying to use torch.autograd.grad

import torch
import metatensor
from metatensor import Labels, TensorMap, TensorBlock

g = torch.Generator()
g.manual_seed(0)
input_values1 = torch.rand(3, 5, generator=g, requires_grad=True)
values1 = input_values1**2 # some operation that makes ϑf(values)/ϑvalues not equal one
values1_grad = torch.autograd.grad(inputs=input_values1, outputs=values1,
        grad_outputs=torch.ones_like(values1), retain_graph=True)[0]

block1 = metatensor.block_from_array(values1.detach())
block1.add_gradient(
    parameter="g",
    gradient=TensorBlock(
        values=values1_grad.detach(),
        samples=Labels.range("sample", len(values1)),
        components=[],
        properties=block1.properties,
    ),
)
tensor1 = TensorMap(Labels.range("_", 1), [block1])

output_tensor = metatensor.add(tensor1, tensor1)
output_values = values1 + values1
grad_autograd = torch.autograd.grad(
        inputs=input_values1,
        outputs=output_values,
        grad_outputs=torch.ones_like(output_values),
        retain_graph=False)[0]
grad_metatensor = output_tensor[0].gradient("g").values
print(torch.allclose(grad_autograd, grad_metatensor))

The hope was by using this util that we do not require our custom finite difference implementation, but since we want that the tests work also for numpy, it does not seem like a good approach.