How to use the trained NN as a function that can be used in the function pde defined in the main function?

[lacrymose]

@lululxvi How to use the trained NN as a function that can be used in the function pde defined in the main function? Many thanks in advance!

[lululxvi]

Maybe not. But I think your problem can be solved by check the "Q: How can I use a trained model for new predictions?" at FAQ.

[lacrymose]

@lululxvi If we want obtain the gradient of the solution, in addition to solution itself, how to accomplish this? Can we directly use tf.gradients(model.predict(x),x) ? If not, what do we do with this case?

[lululxvi]

You can use the following to obtain the gradient at X:

dy = model.predict(X, operator=lambda x, y: tf.gradients(y, x)[0])

[lacrymose]

Many thanks! Can we use multiple networks to solve a pde system by using DeepXDE? For example, if a system of two pdes is given, is it possible to use two networks in DeepXDE to solve this system?

[lacrymose]

these two pdes are tightly coupled. let's say the coefficients of one pde regarding one unkown is a function of the unkown.

[lululxvi]

Yes, you can directly choose the output layer of the network to be 2 neurons. See the example ode_system.py

[lacrymose]

Can we use two indivisual networks instead of one network with two outputs to achieve this? I find it hard to solve the system using one network with two outputs due to the difficulty of balancing the losses from pdes, bcs and ics. So I want to use two indivisual networks, each with one output, to solve the system. But I have no idea of how to use DeepXDE to get this done.

[lululxvi]

Currently DeepXDE only uses one network, and you have to modify the code "maps/fnn.py" to use two networks, which is also easy to do. If you want to do this, we can discuss the details.

Using one network or two networks are essentially the same. You just need to use a large network. Of course, you need to tune the weights of each loss term. (I suspect that even if you use two networks, you still need to tune.) For example, you can check the paper "Systems biology informed deep learning for inferring parameters and hidden dynamics" (https://www.biorxiv.org/content/10.1101/865063v2) for the ODE system, where there are 7 very different outputs.

[lacrymose]

I certainly want to learn how to modify the code "maps/fnn.py" to use two network. Tell me how to achieve this, please. Many thanks in advance. My idea to solve this system is to represent each unkown with one network and solve each pde one by one iteratively. By doing this, I can reduce the number of loss terms in each subproblem and thus alleviate the difficulty of balancing the losses in the loss functions. Although the time spending on iterations might be large, solving each subproblem easier. One possible issue might be that using float32 is not sufficient to ensure convergent solutions come out. What do think of this idea?

[smao-astro]

Hi @lacrymose

I happened to have an implementation (#96) that might be helpful for you, but I can not guarantee that it can improve the performance .

[lululxvi]

@lacrymose Even using multiple networks to represent the unknown, I still prefer to train all the networks simultaneously, instead one by one. Iterative training may requires to modify the code a lot, but more importantly, I am not sure whether iterative training works, or how well it works.

@smao-astro What is your experience about the multiple FNN vs one FNN? Do you observe a better performance?

[smao-astro]

What is your experience about the multiple FNN vs one FNN? Do you observe a better performance?

Hi @lululxvi , I have not conduct any test that can give a clear answer to the question, however I do interested in that, if anyone would like to do the test, I would be appreciated.

By the way, I do see people do so, see paper which uses 3 sub-networks

Sun, L., Gao, H., Pan, S., Wang, J. (2019). Surrogate Modeling for Fluid Flows Based on Physics-Constrained Deep Learning Without Simulation Data arXiv 361(), 112732. https://dx.doi.org/10.1016/j.cma.2019.112732

[lululxvi]

Yes, I know some work does this, e.g., https://doi.org/10.1016/j.jcp.2019.07.048 . But I am not sure when and how these two differs systematically.

[smao-astro]

Neither do I, I am just worrying that if the last hidden layer is shared by, lets say three outputs (hereafter y), is it that there is an linear-like relationship between these outputs, for example, if you imagine the outputs from the last hidden layer as the basis of a space, isn't that the y are like different planes in the same space? Some thought like that encouraged me to add nonlinear difference to the outputs.

[lululxvi]

Theoretically, the linearity is not a problem. For example, we can use polynomial basis as a comparison: let e1(x), e2(x), ... be a set of polynomial bases, and any continuous function can be written as c1 e1 + c2 e2 + ..., i.e., only linear summation. But in practice, the nonlinear part may be more efficient.

[lacrymose]

@smao-astro @lululxvi Many thanks!!! If one wants to use multiple networks to solve a system of pdes which are all coupled, for example in my case, one unknown happens to be one of the coefficients of the other pde, I wonder how to embed the network corresponding to the first unknown into the other pde function which is going to be called when training the other network corresponing to the second unknown, or how to call the first network when traing the second one?

[lululxvi]

It is exactly the same as using one network, as shown in ode_system.py

[lacrymose]

@lululxvi Thanks for your response. But I donot understand. In my view, one network with two outputs represent y1 and y2 in ode_system.py. Hence, the ode system can be defined as it does in ode_system.py. While if one wants to use two networks to represent y1 and y2 respectively, it maybe should look like the following def ode1(x, y, y2): \ y1 = y[:, 0:1] \ dy1_x = tf.gradients(y1, x)[0] \ return dy1_x - y2 \ def ode2(x, y, y1): \ y2 = y[:, 0:1] \ dy2_x = tf.gradients(y2, x)[0] \ return dy2_x + y1 I am not sure if this is right in terms of python syntax. What do you think?

[lululxvi]

No. One network and multiple networks have the same interface. This is guaranteed by the implementation inside DeepXDE.

[lacrymose]

Could you give me a code example to achieve this, please? I have no idea of how to do this, even with ode_system.py. Could you give more details?

[lululxvi]

ode_system.py uses one network for two outputs. If you want to use two networks, and then replace net = dde.maps.FNN(...) with net = dde.maps.PFNN(...) and use layer_size = [1] + [[10, 30], [20, 40]] + [2] to define the two networks (the first network has two hidden layers, and the first hidden layer has 10 neurons, and the second hidden layer has 20 neurons; the second network has two hidden layers, with 30 and 40 neurons). Everything else remains the same.

Of course, you need to wait the pull request https://github.com/lululxvi/deepxde/pull/96 to merge into DeepXDE.

[lululxvi]

The multiple network is in DeepXDE.

[zxw4688]

You can use the following to obtain the gradient at X:

dy = model.predict(X, operator=lambda x, y: tf.gradients(y, x)[0])

Thank you very much for your providing me to utilize the the function of FNN in training, if I would like to obtain the second partial derivative on x[:,0:1] , x[:,1:2] and so on, I think I can perform as: dy_x, dy_t = dy_x[:, 0:1], dy_x[:, 1:] dy_xx = model.predict(X, operator=lambda x, y: tf.gradients(dy_x, x)[0] [:, 0:1]) dy_tt = model.predict(X, operator=lambda x, y: tf.gradients(dy_t, x)[0] [:, 1:])

Is it correct?

[lululxvi]

def f(x, y):
    dy_x = tf.gradients(y, x)[0][:, 0:1]
    dy_xx = tf.gradients(dy_x, x)[0][:, 0:1]
    return dy_xx

dy_xx = model.predict(X, operator=f)

[zxw4688]

Thank you very much !

[zxw4688]

Hello @lululxvi 你好！ 我在使用Deepxde求解inverse问题的时候，我第一次给了pde的参数一个初值进行训练，但是训练结束后我发现不是很好，我现在重新设定一个初值，但是此时的权重和偏置的初值，我想把它设定为刚才训练好的权重和偏置，这个怎么设定啊

[lululxvi]

See FAQ "Q: How can I save a trained model and then load the model later?"