The current implementation of the Integral Kernel layer only manages to learn the target function when the target space is discretized by only one value (shape(y) == (batch-size, 1, y-dim)) as it is the case in examples/selfsupervised.ipynb. However, when the target space is discretized with a value larger than 1, the operator fails to learn the target function (at least for specific examples).
Which issue does this PR tackle?
New implementation of Neural Network Kernel is more general and fixes the issue.
How does it solve the problem?
The Neural Network Kernel of the previous implementation was constructed as:
$k(x_i, y_j) = NN\left( (x_i - yj)^2 \right)$.
The issue described above was due to the symmetric properties of the example combined with the specific construction of the kernel. The input function in basics.ipnyb has the property $u(-x) = -u(x)$. One can show that this property combined with the construction of our kernel leads to:
$v{pred}(y_j) = \frac{1}{N} \sum_i k(x_i, y_j) u(xi) = -v{pred}(-y_j)$. Therefore, the operator could only properly learn mappings from asym. to asym. functions (or sym. to sym.).
The new kernel implementation is more general and does not rely on a convolution ($x_i - y_i$).
Note
I had to change one of the test cases because I changed the way the IntegralKernel method processes the inputs x and y. Previously extra dimensions were added via .squeeze(). I deleted these lines. The idea is that the reshaping is the responsibillity of the given kernel and not the IntegralKernel operation.
How are the changes tested?
Tested via tests/test_neuraloperator.py
Tested via tests/test_convolution.py
Checklist for Contributors
[x] Scope: This PR tackles exactly one problem.
[ ] Conventions: The branch follows the feature/title-slug convention.
[ ] Conventions: The PR title follows the Bugfix: Title convention.
[x] Coding style: The code passes all pre-commit hooks.
[x] Documentation: All changes are well-documented.
[x] Tests: New features are tested and all tests pass successfully.
[x] Changelog: Updated CHANGELOG.md for new features or breaking changes.
[x] Review: A suitable reviewer has been assigned.
Checklist for Reviewers:
[x] The PR solves the issue it claims to solve and only this one.
[x] Changes are tested sufficiently and all tests pass.
Bugfix: Problem with kernel integral architecture
Description
The current implementation of the Integral Kernel layer only manages to learn the target function when the target space is discretized by only one value (shape(y) == (batch-size, 1, y-dim)) as it is the case in examples/selfsupervised.ipynb. However, when the target space is discretized with a value larger than 1, the operator fails to learn the target function (at least for specific examples).
Which issue does this PR tackle?
How does it solve the problem?
Note
I had to change one of the test cases because I changed the way the IntegralKernel method processes the inputs x and y. Previously extra dimensions were added via
.squeeze(). I deleted these lines. The idea is that the reshaping is the responsibillity of the given kernel and not the IntegralKernel operation.How are the changes tested?
Checklist for Contributors
feature/title-slugconvention.Bugfix: Titleconvention.Checklist for Reviewers: