Inverse problem for the fractional Poisson equation in 1D cannot be trained by L-BFGS

[xuelanghanbao]

Hi, @lululxvi, when I try to train the example Inverse problem for the fractional Poisson equation in 1D by L-BFGS with dynamic auxiliary points, there's an unexpected error:

File "envs\lib\site-packages\deepxde\model.py", line 348, in closure total_loss.backward() File "envs\lib\site-packages\torch_tensor.py", line 487, in backward torch.autograd.backward( File "envs\lib\site-packages\torch\autograd__init__.py", line 200, in backward Variable._execution_engine.run_backward( # Calls into the C++ engine to run the backward pass RuntimeError: Trying to backward through the graph a second time (or directly access saved tensors after they have already been freed). Saved intermediate values of the graph are freed when you call .backward() or autograd.grad(). Specify retain_graph=True if you need to backward through the graph a second time or if you need to access saved tensors after calling backward.

I tested the example by L-BFGS with static auxiliary points and it worked properly. Here is my code, and my backend is pytorch:

import deepxde as dde
import numpy as np
import deepxde.backend as bkd
from scipy.special import gamma

alpha0 = 1.8
alpha = dde.Variable(1.5)

# Backend tensorflow.compat.v1
def fpde(x, y, int_mat):
    """(D_{0+}^alpha + D_{1-}^alpha) u(x)"""
    if isinstance(int_mat, (list, tuple)) and len(int_mat) == 3:
        int_mat = bkd.sparse_tensor(*int_mat)
        lhs = bkd.sparse_dense_matmul(int_mat, y)
    else:
        lhs = bkd.matmul(int_mat, y)
    lhs /= 2 * bkd.cos(alpha * np.pi / 2)
    rhs = gamma(alpha0 + 2) * x
    return lhs - rhs[: bkd.size(lhs)]

def func(x):
    return x * (np.abs(1 - x**2)) ** (alpha0 / 2)

geom = dde.geometry.Interval(-1, 1)

observe_x = np.linspace(-1, 1, num=20)[:, None]
observe_y = dde.icbc.PointSetBC(observe_x, func(observe_x))

data = dde.data.FPDE(
    geom,
    fpde,
    alpha,
    observe_y,
    [100],
    meshtype="dynamic",
    num_domain=20,
    anchors=observe_x,
    solution=func,
    num_test=100,
)

net = dde.nn.FNN([1] + [20] * 4 + [1], "tanh", "Glorot normal")
net.apply_output_transform(lambda x, y: (1 - x**2) * y)

model = dde.Model(data, net)

model.compile(
    "L-BFGS", lr=1e-3, loss_weights=[1, 100], external_trainable_variables=[alpha]
)
variable = dde.callbacks.VariableValue(alpha, period=1000)
losshistory, train_state = model.train()
dde.saveplot(losshistory, train_state, issave=True, isplot=True)

[lululxvi]

This example is not fully supported by pytorch.

[yuluoxiansen]

This example is not fully supported by pytorch.

Hi, @lululxvi ,In the code fractional_Poisson_2d.py, I changed the geom, but it doesn't work anymore, does it only support disk?

geom = dde.geometry.Rectangle((0, 0), (r, r))

this is an error： Traceback (most recent call last): File "D:\PINNs\CTR_KLK\fpde\train_fpde_2d.py", line 64, in <module> data = dde.data.FPDE( File "D:\ProgramData\anaconda3\envs\tf\lib\site-packages\deepxde\data\fpde.py", line 88, in __init__ super().__init__( File "D:\ProgramData\anaconda3\envs\tf\lib\site-packages\deepxde\data\pde.py", line 127, in __init__ self.train_next_batch() File "D:\ProgramData\anaconda3\envs\tf\lib\site-packages\deepxde\data\fpde.py", line 162, in train_next_batch self.frac_train = Fractional(self.alpha, self.geom, self.disc, x_f) File "D:\ProgramData\anaconda3\envs\tf\lib\site-packages\deepxde\data\fpde.py", line 370, in __init__ self._check_dynamic_stepsize() File "D:\ProgramData\anaconda3\envs\tf\lib\site-packages\deepxde\data\fpde.py", line 377, in _check_dynamic_stepsize min_h = self.geom.mindist2boundary(self.x0) File "D:\ProgramData\anaconda3\envs\tf\lib\site-packages\deepxde\geometry\geometry.py", line 28, in mindist2boundary raise NotImplementedError( NotImplementedError: Rectangle.mindist2boundary to be implemented

[hannanmustajab]

@lululxvi Hey, I am also using Pytorch. What do you mean by not supported ? I am training using Adam on 2D Wave eq (inverse case), and it doesn't seem to work. Can you please clarify.

[lululxvi]

https://github.com/lululxvi/deepxde/blob/85920299331bd7c0bad01f3d2abba442a77c89c6/examples/pinn_forward/fractional_Poisson_2d.py#L1