Open mariaade26 opened 10 months ago
Using
sensealg = SciMLSensitivity.ForwardDiffSensitivity()
in the first two solve
calls may fix this issue.
UPDATE: only if direct call works and the problem is in AD.
Forward mode would likely be faster for a model of this size, yes.
That said, the real issue is that what's being passed doesn't match the interface. Look at the error message:
(::RNNCell{true})(::Tuple{AbstractMatrix, Tuple{AbstractMatrix}}, ::Any, ::NamedTuple)
This is saying what the input of an RNNCell has to be. Now look at what's passed in:
out, st = nn_model(vcat(u[1], Tnorm(t),phinorm(t), phisunnorm(t)), p, st)
It says it wants a Tuple{AbstractMatrix, Tuple{AbstractMatrix}}
as the first argument, but clearly the first argument is a vector defined by a vcat
, so that's very clearly not following Lux's interface. Look at the documentation here and follow it's input specification:
https://lux.csail.mit.edu/dev/api/Lux/layers#Lux.RNNCell
(@avik-pal)
Optimization not working as soon as Dense layer gets replaced with others (ex. RNN)
So this title doesn't really make sense. Other layers work, there's lots of examples of this, for example https://docs.sciml.ai/DiffEqFlux/dev/examples/mnist_conv_neural_ode/ uses convolutional layers so "oh no nothing other than Dense works" is just false. What is true is that when you change the layer type to something different, the neural network library may require a slightly different input (as is the case with RNNCell
) and you need to modify the ODE definition so that it is being called the way that the neural network library expects it to be called. This has nothing to do with SciML though, this would error outside of the ODE even if calling the neural network directly.
Thank you.
You need to have a batch dimension for the layer to work.
Also, note that your current code doesn't do what you want to do. It always treats the input as the first input in the sequence, which makes using an RNNCell quite pointless. You might want to look at https://lux.csail.mit.edu/dev/api/Lux/layers#Lux.StatefulRecurrentCell.
That being said, you need to turn off the adaptivity of the solver, else stateful RNNs don't make sense since the time is not monotonically increasing.
Hi everyone, I have a problem with my code, in particulat while training the neural network, I've encountered a significant issue when attempting to replace a dense layer with other types of layers. Specifically, I've noticed that when I tried to introduce different layers, such as scalar layers or recurrent layers, my code started generating errors related to automatic differentiation (AD). These errors made it impossible to optimize the model and caused issues during gradient backpropagation. Despite my efforts to address this problem, it seems that AD is not compatible with the new layers, leading to a standstill in training. I'm sure I'm missing something but I don't uderstand it. I would really appreciate some help. I'm attachin the code, the csvs, and the error. Thank you! Internal gains.csv phi heating tutto.csv phisun totale.csv Testerna totale.csv phi heating modificato.csv
Here's is my code:
and the related error: