Issue with Example

[JabirSS]

Hi Christian,

Thanks for sharing your great work! I managed to go through the installation steps of your library but when I try to run the mhdChtMultiRegionInterIsoFoam_mod example, I get an issue related to comms between OF and Elmer

OF sending fluid 0 sending mesh to Elmer ERROR:: OpenFOAM2ElmerSolver: Elmer #7 has 448 elements, OpenFOAM found 0 this of course is repeated for all other partitions of the mesh, all of which say that OpenFOAM found 0.

Is there an updated version of the example? There were some other issues such as a small typo in the file system/mixture/fvSolution and also the following error when trying to convert the Elmer mesh:

Loading the geometry from file 'mesh.grd'.
Loading ElmerGrid file version: 210903
Defining the coordinate system (2-DIM).
Loading 6 subcell limits in X-direction
Loading 6 subcell limits in Y-direction
Subcell limits 2(1): 0.000000e+00 1.000000e-02
The program encountered a major error...
Values for limits should be a growing series, existing

It was easily fixed by flipping the order of the subcell limits 2, however, just curious if an updated version of the example exists that may help with the O2E communication issue.

I am running the recommended versions of OF and Elmer.

Thanks,

Jabir

[c-schubert]

Hi Jabir,

I am sorry there may still be some mistakes inside the repo. I basically pulled all the files out of a working environment, made some renames, changed some things here and there and hoped that it would simply work (and of course it never does 😅). I have never actually tested this again in a clean environment. I updated the installation instructions in the main README, so pls make sure you have done the mentioned fixed to elmer (SOLVER.KEYWORDS) and that you are using the right alias before running the solver. I currently do not have the time to invest too much time into this :( ...

Hope this helps Christian

[JabirSS]

Thanks for your quick reply! I did actually apply the alias and the correct solver keywords, but still got the comms issue. Perhaps it is a mesh issue (I hope). I will try to create a new simpler case and see if it all works. I understand that you have moved on to other things so I will try my best. If I made any additions I will push them and I'd appreciate it if you could have the time to review and commit.

One last thing I would like to consult you with.. My application requires adding the velocity to the variables coming from OpenFoam (for the UxB force). I already added it to one of your non-multiregion solvers and it seems to ber working fine, albeit painfully slow on the Elmer side (tight coupling requires frequent solutions by Elmer which may take 20-100 linear system iterations). Just curious if you had tried to do a similar thing and if you could share any tricks you found helpful to speed things up!

Best regards,

Jabir

[c-schubert]

If you use the mesh files I uploaded I think they should work, at least they are the same I used in some of my calculations.

Have you compiled both elemer and openfoam and elmer with the same mpi versions? I think this is crucial for EOF to work ...

And yes Elmer is very slow in some cases and I also got some nonphysical results from elmer in other cases. But with those couplings the FEM part is probably always a lot slower and uses lot more RAM, at least if the mesh sizes are still in the same size, which makes sense in the case of MP couplings... There have been other approaches using FVM methods to solve the vector potential approach by Beckstein, but I had not dived deep into that, as they seem very specialized on a certain problem at the moment.

I also think looking into MFEM or Sparselizard FEM, which are Finite Element Frameworks especially made for HPC would be interesting, but with picking a FEM framework for the coupling, there are a lot of questions to ask about the proper implementation and capabilities of the framework, which you only can answer if you already know the framework very well or do a lot of testing, and I am not even sure if elmer theoretically fulfills all the requirements, as I had some problems with it...

I also used a selfmade ANSYS Fluent to ANSYS Mechanical APDL coupling procedure for comparison in my work and found that this is while being slower (mainly due to fluent), it is also a lot more stable (no nonphysical results), but there are also some derivations in the coupling and solution procedure from my OF-Elmer approach.

There may be reasons for elmer to procuce these unphysical results, but as I am not an expert in FEM and electromagnetic simulations, so I could not figure it out on a shorthand. But I have made some observations that may help you:

• I use linear elements for the elmer case as they seem to be faster and more stable for my case, but in theory second order elements should be better for electromagnetic calculations
  • with the linear elements the stability of the iterative solver seems to be heavily mesh dependent - in general therefore second order elements should be the better choice, but they did not even work out even for a simple test case for me ...
• Comparing with ANSYS (where I use seconds order elements), in ANSYS the direct solver is the recommended, fastest and most reliable solver for the electromagnetic calculations, in elmer none of the available direct solver will even converge for my case (linear or seconds order mesh) (and also they are not recommended by the solver guide)

So maybe my boundary conditions or general settings are very bad, but I tested a lot of different settings ... Furthermore, I do not find many documented applications of the WhitneyAVHarmonicSolver, which is used here, only simple test cases, so I am very suspicious about its real capabilities ...

From the documentation of the softwares I got they use slightly different approaches, which both rely on the vector potential approach, but I cannot really explain why they behave such different.

If your coupling time step is very low and you do not need implicit coupling, you should be able to use transient WhitneyAVSolver, which should be faster and maybe it is more mature?

It is kind of sad that there is no mature open source software can do these kinds of couplings, but also I think on a commerical side of things the sun only shines very little brighter there, at least not for the coupling between multi phase flow with a finite volume framework and a general and flexible electro-magnetic fem simulation software.

Also I think its hard to get funding for developing such things in a solid way, since the application of such frameworks are restricted to very few applications, at least I do not know many applications which make such approaches really necessary 😅

[JabirSS]

Thanks for sharing your experiences. It is indeed unfortunate that no open-source software has been developed for this, but as you said, the applications are quite niche. I am in fact planning to eventually implement the magnetic vector potential equations in OpenFoam and see if it works well (relative to full EM either in a commercial code or by coupling Elmer to OpenFOAM. Thanks to your work with merging the multiphase solvers with the multi-region solver, my job is a little easier now.

I'll keep working on it and I'll let you know if anything good came out of it.

Best,

Jabir