Question regarding solution type

[RobWalt]

I'm currently trying to understand the whole Neural ODE Framework a bit better. At the moment I feel a bit stuck. I don't really get what the output of a NeuralODE Layer is exactly.

My example code is:

...
down_layer = Chain(Dense(784, 32, tanh))
dense_layers = Chain(Dense(32, 16),
                     Dense(16, 32))
...
nn_ode = NeuralODE(dense_layers,
                  (0.0, 1.0),
                  Tsit5(),
                  reltol=1e-3,
                  abstol=1e-3,
                  save_start=false)
...

with the MNIST data as input. Now when I feed one flat input image step by step through the network, I get

size(down_layer(img)) # = (32,1) ... that's ok
size(nn_ode(down_layer(img))) # = (32, 1, 4) ... ?

I don't really get why the last dimension of this output is 4. I already tried to look it up in the documentation, but all I'm getting is, that the solution type (and form) is defined by the arguments you call solve with, which is called in NeuralODE. But this still doesn't explain to me how to interpret the 4 solution arrays independently.

So my question is: What are the 4 solution arrays for and how can I interpret them?

[ChrisRackauckas]

That's because it's taking 4 steps and saving the results along the time series.

[RobWalt]

Ok, thanks. Quick follow-up question: Then the last entry in the solution array corresponds to the latest version of the approximated solution, right?

[ChrisRackauckas]

It corresponds to the approximation of the solution at the end, yes. It's all as described in https://diffeq.sciml.ai/stable/tutorials/ode_example/

[RobWalt]

Oh, my bad. Thanks again! 🥇