search

precice / calculix-adapter

preCICE-adapter for the CSM code CalculiX
GNU General Public License v3.0
52 stars 20 forks source link

RBF mapping: Polynomial QR linear system has not converged. (specific to v2.16) #37

Closed MakisH closed 3 years ago

MakisH commented 4 years ago

As I wrote on https://github.com/precice/calculix-adapter/pull/33, which is now merged to master:

This time I tried on a clean install on Ubuntu 20.04, preCICE 2.0.1 from master built in Debug mode, PETSc 3.12 from APT (3.12.4+dfsg1-1), CalculiX 2.16 and the adapter from the v2.16 branch.

Running the tutorials/FSI/flap_perp/OpenFOAM-CalculiX tutorial without any changes, with OpenFOAM v1912, returns on the Fluid side:

---[preciceAdapter] [DEBUG] Writing coupling data...
---[preciceAdapter] [DEBUG] Advancing preCICE...
---[precice]  Mapping Forces0 conservative from Fluid-Mesh-Faces (ID 0) to Solid (ID 2) for dimension 0) with polynomial set to separate
---[precice]  Mapping Forces0 conservative from Fluid-Mesh-Faces (ID 0) to Solid (ID 2) for dimension 1) with polynomial set to separate
---[precice]  Mapping Forces0 conservative from Fluid-Mesh-Faces (ID 0) to Solid (ID 2) for dimension 2) with polynomial set to separate
---[precice]  Compute read mapping from mesh "Solid" to mesh "Fluid-Mesh-Nodes".
---[precice]  Using tree-based preallocation for matrix C
---[precice]  Using tree-based preallocation for matrix A
---[precice]  Mapping Displacements0 consistent from Solid (ID 2) to Fluid-Mesh-Nodes (ID 1) for dimension 0) with polynomial set to separate
KSP Object: QR Solver 1 MPI processes
  type: lsqr
    standard error not computed
    using inexact matrix norm
  maximum iterations=10000, initial guess is zero
  tolerances:  relative=1e-05, absolute=1e-50, divergence=10000.
  left preconditioning
  using UNPRECONDITIONED norm type for convergence test
PC Object: 1 MPI processes
  type: none
  linear system matrix = precond matrix:
  Mat Object: Q 1 MPI processes
    type: seqdense
    rows=86, cols=4
    total: nonzeros=344, allocated nonzeros=344
    total number of mallocs used during MatSetValues calls=0
---[precice] ERROR:  Polynomial QR linear system has not converged. Try to fix axis-aligned mapping setups by marking perpendicular axis as dead.

Again, running the same exact tutorial on the v2.15 is perfectly fine.

I got the same issue with the FSI/3DTube/OpenFOAM-CalculiX tutorial (switching to rbf-thin-plate-splines mapping). I also observed it on Ubuntu 16.04 with PETSc 3.9.1 built from source.

It looks to me that this is specific to the CalculiX adapter for v2.16, but I have no clue why.

precice-bot commented 4 years ago

This issue has been mentioned on preCICE Forum on Discourse. There might be relevant details there:

https://precice.discourse.group/t/influence-of-the-number-of-processors-in-cylinderflap-tutorial/263/2

DamienGuenot commented 4 years ago

As suggested by @MakisH on preCICE forum, I am updating my findings on the same issue. Here are my main dependancies

With CalculiX v2.16 and CalculiX adapter for v2.16, I get the following results with the tutorials:

besserox commented 4 years ago

As commented here https://precice.discourse.group/t/influence-of-the-number-of-processors-in-cylinderflap-tutorial/263/5,

I also ran into the same issue while trying the flap_perp tutorial. Downgrading to CalculiX 2.15 worked.

Eder-K commented 4 years ago

Commenting here as well to mention that the systemtest for FSI/flap_perp/OpenFOAM-CalculiX is failing as of recently:

calculix-adapter-solid    | Adapter calling advance()...
openfoam-adapter-fluid    | ---[precice]  Using tree-based preallocation for matrix A
openfoam-adapter-fluid    | ---[precice]  Mapping Displacements0 consistent from Solid (ID 2) to Fluid-Mesh-Nodes (ID 1) for dimension 0) with polynomial set to separate
openfoam-adapter-fluid    | ---[precice]  Mapping Displacements0 consistent from Solid (ID 2) to Fluid-Mesh-Nodes (ID 1) for dimension 1) with polynomial set to separate
openfoam-adapter-fluid    | [0]PETSC ERROR: ------------------------------------------------------------------------
openfoam-adapter-fluid    | [0]PETSC ERROR: Caught signal number 8 FPE: Floating Point Exception,probably divide by zero
openfoam-adapter-fluid    | [0]PETSC ERROR: Try option -start_in_debugger or -on_error_attach_debugger
openfoam-adapter-fluid    | [0]PETSC ERROR: or see http://www.mcs.anl.gov/petsc/documentation/faq.html#valgrind
openfoam-adapter-fluid    | [0]PETSC ERROR: or try http://valgrind.org on GNU/linux and Apple Mac OS X to find memory corruption errors
openfoam-adapter-fluid    | [0]PETSC ERROR: configure using --with-debugging=yes, recompile, link, and run 
openfoam-adapter-fluid    | [0]PETSC ERROR: to get more information on the crash.
openfoam-adapter-fluid    | [0]PETSC ERROR: User provided function() line 0 in  unknown file  
openfoam-adapter-fluid    | --------------------------------------------------------------------------

(Check the linked TravisCI build for the full log)

This test uses CalculiX v2.16 and the respective develop-branch versions of the adapters and tutorial.

TreeDiagramist commented 4 years ago

Nice to meet you. I'm pleased to report that I have completed the tutorial using CalculiX-Adapter with CalculiX 2.16. I am working on Ubuntu 20.04 LTS. The versions of each software and library are as follows. ・PreCICE 2.1.0 ・CalculiX 2.16 ・Boost 1.73.0 ・PETSc 3.13.3 ・OpenMPI 4.0.4 I built the above and used the apt command to install other dependencies. The following two tutorials have been confirmed to work in my environment. ・FSI/flap_perp/OpenFOAM-CalculiX ・SSI/loaded_beam/CalculiX-CalculiX cylinderFlap cannot be calculated because an error occurred on the OpenFOAM-Adapter side. I don't know about 3D_Tube because I haven't done it yet. In the future, I plan to check the reproducibility using the same operation method. Also, if there is anything, I will report it.

MakisH commented 4 years ago

Thank you very much, @TreeDiagramist!

cylinderFlap cannot be calculated because an error occurred on the OpenFOAM-Adapter side.

Is maybe the error similar to the one here?

ERROR:  Polynomial QR linear system has not converged.

If not, I would be happy to help you with this on Gitter or Discourse.

TreeDiagramist commented 4 years ago

@MakisH The log file of OpenFOAM-Adapter of cylinderFlap is attached. As you can see from the log file, the error that is occurring in my environment does not seem to be the same.

P.S. The tutorial of 3D_Tube was able to confirm the execution with the Allrun file. runFluid.log

MakisH commented 4 years ago

@TreeDiagramist this I fixed already, but only in develop. Simply remove or adjust residualControl in your Fluid/system/fvSolution file.

TreeDiagramist commented 4 years ago

@MakisH Thank you for your advice! After editing Fluid/system/fvSolution, I was able to confirm the execution of cylinderFlap.

MakisH commented 4 years ago

I get a very similar error with our SU2-CalculiX tutorial, which again appears with CalculiX 2.16 but not with 2.15 (and the respective adapter version).

Log files:

SU2 ``` ---[precice] Compute read mapping from mesh "Calculix_Mesh" to mesh "SU2_Mesh0". ---[precice] Started event advance/map.nn.computeMapping.FromCalculix_MeshToSU2_Mesh0 ---[precice] Compute consistent mapping ---[precice] Started event advance/map.nn.computeMapping.FromCalculix_MeshToSU2_Mesh0.getIndexOnVertices ---[precice] Stopped event advance/map.nn.computeMapping.FromCalculix_MeshToSU2_Mesh0.getIndexOnVertices ---[precice] Mapping distance min:0 max:1.66439e-12 avg: 5.77566e-13 var: 3.34676e-25 cnt: 24 ---[precice] Stopped event advance/map.nn.computeMapping.FromCalculix_MeshToSU2_Mesh0 ---[precice] Map read data "DisplacementDeltas0" to mesh "SU2_Mesh0" ---[precice] Started event advance/map.nn.mapData.FromCalculix_MeshToSU2_Mesh0 ---[precice] Map consistent ---[precice] Stopped event advance/map.nn.mapData.FromCalculix_MeshToSU2_Mesh0 ---[precice] Mapped values = [0, 0, 0, ... , 0, 0, 0] min:0 max:3.47091e-310 ---[precice] it 2 of 50 | dt# 1 of 400 | t 0 | dt 0.01 | max dt 0.01 | ongoing yes | dt complete no | read-iteration-checkpoint | ---[precice] Handle exports ---[precice] Started event solver.advance ---[precice] Stopped event advance Deforming the volume grid due to preCICE simulation. Computing volumes of the grid elements. Min. volume: 0.00113526, max. volume: 0.111064. Error in "void CSysSolve::ModGramSchmidt(int, std::vector >&, std::vector&)": ------------------------------------------------------------------------- SU2 has diverged. ------------------------------ Error Exit ------------------------------- ```
CalculiX ``` Adapter calling advance()... ---[precice] Stopped event solver.advance ---[precice] Started event advance ---[precice] Synchronize timestep length ---[precice] return 0 ---[precice] return 0 ---[precice] return 0 ---[precice] Advance coupling scheme ---[precice] return 0 ---[precice] Receiving data... ---[precice] Started event advance/m2n.receiveData ---[precice] Stopped event advance/m2n.receiveData ---[precice] Number of received data sets = 1 ---[precice] Perform acceleration (only second participant)... ---[precice] measure convergence of the coupling iteration ---[precice] relative convergence measure: relative two-norm diff = -nan, limit = 0.001, normalization = 0.000593277, conv = false ---[precice] relative convergence measure: relative two-norm diff = 0.5, limit = 0.001, normalization = 15.4184, conv = false ---[precice] Started event advance/cpl.computeQuasiNewtonUpdate ---[precice] Update Difference Matrices ---[precice] Performing quasi-Newton Step ASSERTION FAILED Location: virtual void precice::acceleration::impl::ResidualSumPreconditioner::_update_(bool, const VectorXd&, const VectorXd&) File: /home/makish/github/precice_develop/src/acceleration/impl/ResidualSumPreconditioner.cpp:48 Rank: 0 Failed expression: sum > 0 0# getStacktrace[abi:cxx11]() in /home/makish/github/precice/build/libprecice.so.2 1# precice::acceleration::impl::ResidualSumPreconditioner::_update_(bool, Eigen::Matrix const&, Eigen::Matrix const&) in /home/makish/github/precice/build/libprecice.so.2 2# precice::acceleration::impl::Preconditioner::update(bool, Eigen::Matrix const&, Eigen::Matrix const&) in /home/makish/github/precice/build/libprecice.so.2 3# precice::acceleration::BaseQNAcceleration::performAcceleration(std::map, std::less, std::allocator > > >&) in /home/makish/github/precice/build/libprecice.so.2 4# precice::cplscheme::BaseCouplingScheme::accelerate() in /home/makish/github/precice/build/libprecice.so.2 5# precice::cplscheme::ParallelCouplingScheme::exchangeDataAndAccelerate() in /home/makish/github/precice/build/libprecice.so.2 6# precice::cplscheme::BaseCouplingScheme::advance() in /home/makish/github/precice/build/libprecice.so.2 7# precice::impl::SolverInterfaceImpl::advance(double) in /home/makish/github/precice/build/libprecice.so.2 8# precice::SolverInterface::advance(double) in /home/makish/github/precice/build/libprecice.so.2 9# precicec_advance in /home/makish/github/precice/build/libprecice.so.2 10# 0x00000000005DCB67 in ccx_preCICE 11# 0x00000000005CB689 in ccx_preCICE 12# 0x000000000040B884 in ccx_preCICE 13# __libc_start_main in /lib/x86_64-linux-gnu/libc.so.6 14# 0x0000000000413E59 in ccx_preCICE ccx_preCICE: /home/makish/github/precice_develop/src/acceleration/impl/ResidualSumPreconditioner.cpp:48: virtual void precice::acceleration::impl::ResidualSumPreconditioner::_update_(bool, const VectorXd&, const VectorXd&): Assertion `false' failed. ```
IshaanDesai commented 3 years ago

In the latest tutorials release and the latest CalculiX-Adapter release this issue is no longer observed