Problem with kernel integral architecture

[MLuchmann]

Description

The current implementation of the Integral Kernel layer only manages to learn the target function when the target space is discretized by only one value (shape(y) == (batch-size, 1, y-dim)) as it is the case in examples/selfsupervised.ipynb. However, when the target space is discretized with a value larger than 1, the operator fails to learn the target function.

To reproduce the problem see basics.ipynb in the linked branch. I replaced the DeepONet with a single Integral Kernel layer (the problem is still there when we use more than one layer such as the current NeuralOperator implementation). The trained model converges to a constant function which coincides with the average of the target function (mean(v(y))). This is independent of the complexity of the model and the number of epochs.

[MLuchmann]

One thing to add: When changing the data range in the example provided in basics.ipnyb to: torch.linspace(-L*1.5, L, ...) for x and y the kernel starts learning something useful. In addition, when changing the example from sin and cosine functions to learn asymmetric functions such as a map from the u(x)=x**3 to x v(y)= y, the kernel operator again manages to learn the mapping successfully.