Solving non-linear diffusion equation

[Sunny-DV]

Hi, I saw your code which is very helpful. Currently, I am solving 1D non-linear diffusion equation. I have successfully solved linear equation suing PINN model. For non-linear equation when i am training my network , it is showing Nan value after 1 iteration. I am using this piece of code for internal data.

def loss_initernal(self, x_train):
    g = x_train.clone()
    g.requires_grad = True
    u = self.forward(g)
    u_g = gradients(u, g)[0]
    u_x, u_t= u_g[:, [0]], u_g[:, [1]]
    D = 1.1*10**(-9)*(self.forward(g))**1.2

    u_xx = gradients(D, g)[0]
    u_xx = u_xx[:, [0]]
    pde = u_t - u_xx
    loss_pde = pde.pow(2).mean()
    print("D",D,"u_x",u,'loss', loss_pde)
    return loss_pde 

def gradients(outputs, inputs):
         return torch.autograd.grad(outputs, inputs, grad_outputs=torch.ones_like(outputs), create_graph=True)

While running this nan is coming after 1 iteration. Can you tell me what i am doing wrong.

[omniscientoctopus]

Hello Sunny,

TL;DR Upon forwardprop self.forward(g) can return negative values, which is expected behaviour. In expression D = 1.1*10**(-9)*(self.forward(g))**1.2, the square root of the negatives, i.e. (self.forward(g))**1.2 return nan instead of a complex number. Since any operation on nan is also nan, all following outputs are also nan

I have reproduced the issue as follows:

import torch
import torch.autograd as autograd         # computation graph
from torch import Tensor                  # tensor node in the computation graph
import torch.nn as nn                     # neural networks

import numpy as np

Create sample points in 1D space and time

x = np.linspace(-1, 1, 6).reshape((-1,1))
t = np.linspace(0, 10, 6).reshape((-1,1))

X_f_train = np.hstack((x, t))

X_f_train = torch.from_numpy(X_f_train).float()

Define PINN class with your PDE

class Sequentialmodel(nn.Module):

    def __init__(self,layers):
        super().__init__() #call __init__ from parent class 

        'activation function'
        self.activation = nn.Tanh()

        'Initialise neural network as a list using nn.Modulelist'  
        self.linears = nn.ModuleList([nn.Linear(layers[i], layers[i+1]) for i in range(len(layers)-1)])

        self.iter = 0

        'Xavier Normal Initialization'
        # std = gain * sqrt(2/(input_dim+output_dim))
        for i in range(len(layers)-1):

            # weights from a normal distribution with 
            # Recommended gain value for tanh = 5/3?
            nn.init.xavier_normal_(self.linears[i].weight.data, gain=1.0)

            # set biases to zero
            nn.init.zeros_(self.linears[i].bias.data)

    'foward pass'
    def forward(self,x):

        if torch.is_tensor(x) != True:         
            x = torch.from_numpy(x)                

        #convert to float
        a = x.float()

        for i in range(len(layers)-2):

            z = self.linears[i](a)

            a = self.activation(z)

        a = self.linears[-1](a)

        return a

    def loss_initernal(self, x_train):

        g = x_train.clone()
        g.requires_grad = True
        u = self.forward(g)
        u_g = self.gradients(u, g)[0]
        u_x, u_t = u_g[:, [0]], u_g[:, [1]]
        D = 1.1*10**(0)*(self.forward(g))**1.2

        print("u & D: \n", torch.hstack((self.forward(g), D)))

        u_xx = self.gradients(D, g)[0]
        u_xx = u_xx[:, [0]]
        pde = u_t - u_xx

        loss_pde = pde.pow(2).mean()
        # print("D",D,"u_x",u,'loss', loss_pde)
        return loss_pde 

    def gradients(self, outputs, inputs):

            return torch.autograd.grad(outputs, inputs, grad_outputs=torch.ones_like(outputs), create_graph=True)

layers = np.array([2,20,20,1])

PINN = Sequentialmodel(layers)

print("PDE loss:", PINN.loss_initernal(X_f_train))

One instance of the output where some u values are negative and corresponding values of D are nan

>>> u & pde: 
 tensor([[-0.2780,     nan],
        [ 0.4945,  0.0398],
        [ 0.6356,  0.0392],
        [ 0.6074,  0.0622],
        [ 0.5635,  0.0711],
        [ 0.5237,  0.0684]], grad_fn=<CatBackward>)
>>> PDE loss: tensor(nan, grad_fn=<MeanBackward0>)

I hope this answers your query.

[Sunny-DV]

Thank a lot.