Troublesome NaNs in optimizations

[Alvf]

Hello!

I'm using PhiFlow with the torch backend to optimize a sequence of control forces in a fluid solver to match sequential frames in a hand-created morph animation, and I'm running into various issues.

It seems that for short sequences of frames, optimization finds a passable set of control forces, but for longer sequences of frames, the optimizer begins to throw NaNs and other errors associated with gradients of make_incompressible() and other things (found using torch.autograd.set_detect_anomaly(True)).

I've attached a minimal example of the issue I'm running into (using randomized tensor sequences just to keep things compact); any advice would be greatly appreciated. minimal_err.txt

Many thanks! Phiflow is quite enjoyable to use.

[holl-]

Hi, thanks for the feedback!

You can do a couple of things.

• Use math.print_gradient() to see what the gradient is at different stages of your graph.
• Use a math.SolveTape() to record the linear solves from the backward pass.
• Use phi.set_logging_level() to enable debug prints.

Hope that helps! Let me know if the error persists.