mlech26l / ncps

PyTorch and TensorFlow implementation of NCP, LTC, and CfC wired neural models
https://www.nature.com/articles/s42256-020-00237-3
Apache License 2.0
1.94k stars 311 forks source link

LtC and CfC implementation questions #57

Open dimarkov opened 10 months ago

dimarkov commented 10 months ago

I am trying to implement LtC and CfC models in Jax. However, after going through the code I found some differences between what was discussed in related publications and what was implemented here. I hope someone could clarify this for me here.

First, in the LtC implementation it seems that nominator and denominator depend on a different nonlinear mapping 'f'. In the nominator one uses an adjacency matrix which has both positive and negative connections and in denominator the adjacency matrix is strictly positive. I understand that one requires this for scaling constant to be always positive, but I am wondering why the choice was made to have effectively two different functions. Given the implementation the actual model should be written as

$$ \dot{x} = - \left(\frac{1}{\tau} + f_1(x, I) \right) x + f_2(x, I)A$$

where only $f_1$ is a strictly positive nonlinear mapping.

Second, given this choice for LtC implementation I am not sure how can one use fitted weights from LtC model inside CfC model. to my understanding for the CfC solution to work the equivalent ODE expression should be

$$ \dot{x} = - \left(\frac{1}{\tau} + f(x, I) \right) x + \left( \frac{1}{\tau} + f(x, I) \right)A $$

where $f$ has to be identical in both terms.

This leads me to the final question, that is of the subtle differences in terms of fix points of this two ODEs

$$ \dot{x} = - \left( \frac{1}{\tau} + f(x, I) \right) x + \left( \frac{1}{\tau} + f(x, I) \right) A $$

and

$$ \dot{x} = - \left(\frac{1}{\tau} + f(x, I) \right) x + f(x, I) A $$

In the first ODE $x^* = A$ is a fixed point solution, which one can also see from CfC solution. In the second equation the fix point depends properly on $f(x, I)$ and to me this seems as more flexible model. However, I am then a bit confused with the claim that the CfC solution provides the same flexibility and is equivalent to the second ODE, rather then the first one.

I would highly appreciate some clarification on this issues.