search

SciML / NeuralPDE.jl

Physics-Informed Neural Networks (PINN) Solvers of (Partial) Differential Equations for Scientific Machine Learning (SciML) accelerated simulation
https://docs.sciml.ai/NeuralPDE/stable/
Other
976 stars 195 forks source link

Approximate a given function #182

Closed linwaytin closed 3 years ago

linwaytin commented 3 years ago

First thanks for this package. Solving PDEs using neural networks is an interesting approach.

I would like to see if a neural network can approximate a given function like sin(x), so I build a simple network

m = Chain(Dense(1, 8, σ),
                 Dense(8, 8, σ),
                 Dense(8, 8, σ),
                 Dense(8, 1))

Gradient descent is used to minimize the 2-norm between sin(x) and m(x). The points x are taken uniformly between -2 and 2. I am surprised that the parameters blow up.

Could anyone please give some advices on this issue? I thought for a simple function like sin, the result should be convergent easily. Did I miss any important point? Thanks.

ChrisRackauckas commented 3 years ago

I know you can easily do something like sin because look at these lecture notes that do it in differential equation form:

https://mitmath.github.io/18337/lecture3/sciml.html

I can't help you without seeing code. However, this is orthogonal to NeuralPDE.jl itself, so I'm going to close this issue since there's nothing to do. If you want to have a wider discussion on this topic, I suggest checking out the Julia Discourse (discourse.julialang.org) or the Julia Slack (julialang.org/slack) which is more for these kinds of open discussions on mathematical and numerical topics.

linwaytin commented 3 years ago

Thanks! I didn't know the existence of the lecture notes. I will go through them and ask questions on Julia Discourse.