Open Mustardburger opened 1 year ago
@Mustardburger @DesmondYuan Some thoughts:
lambda x: x ** k / (1 + torch.abs(x) ** k)
do you know at what specific point there starts being an error? Is x**k
or 1 + torch.abs(x) ** k
or something else?
@cannin @DesmondYuan:
I am also thinking, since both tensorflow and pytorch code right now are identical, but lead to different solutions, which one of them is the more correct solution?
Issue type
Bug or help needed
Relevant package versions
numpy == 1.24.1 tensorflow == 2.11.0 torch == 2.0.1
Python version
3.8.0
Current behaviour
The envelope forms in tensorflow and pytorch (defined here) yield very similar results (their difference between the two outputs is on the scale of 10e-8). However, these differences accumulate after several time steps in the ODE solver, and become very noticeable after around 150 to 200 time steps in the solver.
Code to reproduce
The recommended envelope form for CellBox is the tanh. The code below calculates the output from tensorflow's and pytorch's isolated envelope form set to tanh (defined in
KernelConfig
). There is no ODE involved yet.The output is:
If using
polynomial
withargs.polynomial_k = 2
:The output is:
However, if changing the envelope form to
clip linear
:The output is:
This difference might be small, but it adds up within the ODE solver, and causes the final result of the tensorflow and pytorch ODE solver to differ significantly. The same issue persisted when
args.envelope_form
is set tohill
orpolynomial
. However, whenargs.envelope_form
is set tolinear
orclip linear
, the difference between tensorflow and pytorch ODE solver is exactly 0, leading me to believe the numerical discrepancy of the other envelope functions cause this behaviour.Solution
Is there a way around this? If two ODE solutions are very different, which one is the correct solution?