search

DLR-AMR / t8code

Parallel algorithms and data structures for tree-based AMR with arbitrary element shapes.
https://dlr-amr.github.io/t8code/
GNU General Public License v2.0
124 stars 48 forks source link

Advection example with pyramids #390

Open sandro-elsweijer opened 1 year ago

sandro-elsweijer commented 1 year ago

I have tried to run the advection example with a hybrid mesh consisting of hexes, tets and pyramids. This results in a whole cascade of asserts to trigger. Maybe someone with more knowledge about ghosts and pyramids can have a look at this. I have created a minimal testing example: cube.zip I have tried to run it with: mpirun -np 8 ./t8_advection -u7 -l0 -r1 -f cube -d3 -C0.5 -b4 -X-0.7 -Y0 -Z0 -R0.3 -T1 in debug mode.

holke commented 1 year ago

Fixed some issues in #391 but still WIP.

Would be great to have a smaller example.

holke commented 1 year ago

@Davknapp will help

lukasdreyer commented 1 year ago

@Davknapp Did you verify, that this is now fixed with the pyramid volume bugfix?

Davknapp commented 1 year ago

@Davknapp Did you verify, that this is now fixed with the pyramid volume bugfix?

I only checked it for two different examples, where it worked. For a safe integration into the main branch, we should test it more carefully.

sandro-elsweijer commented 1 year ago

I will test it as well with the example where I found the bug.

sandro-elsweijer commented 1 year ago

The volume bug definitely had something to do with the advection example not working with pyramids. But it still does not work. Continuing the discussion in #391

holke commented 1 year ago

Update: #391 should solve some of the pyramid issues. Advection is working with example meshes.

However, we still run into problems with the cube mesh.

mpirun -np 8 --oversubscribe ./t8_advection -u7 -l0 -r1 -f cube -d3 -C0.5 -b4 -X-0.7 -Y0 -Z0 -R0.3 -T1

produces

[libsc 6] Abort: Assertion 'element_index >= 0'
[libsc 6] Abort: ./../t8code/src/t8_forest/t8_forest_cxx.cxx:1932
[libsc 6] Abort: Obtained 10 stack frames
[libsc 6] Stack 0: t8_advection(+0xc97a5) [0x558f0443b7a5]
[libsc 6] Stack 1: t8_advection(+0xc9760) [0x558f0443b760]
[libsc 6] Stack 2: t8_advection(+0xc9981) [0x558f0443b981]
[libsc 6] Stack 3: t8_advection(+0x5c539) [0x558f043ce539]
[libsc 6] Stack 4: t8_advection(+0x18a3c) [0x558f0438aa3c]
[libsc 6] Stack 5: t8_advection(+0x1963e) [0x558f0438b63e]
[libsc 6] Stack 6: t8_advection(+0x1b256) [0x558f0438d256]
[libsc 6] Stack 7: libc.so.6(+0x29d90) [0x7f6bd6891d90]
[libsc 6] Stack 8: libc.so.6(__libc_start_main+0x80) [0x7f6bd6891e40]
[libsc 6] Stack 9: t8_advection(+0x15d25) [0x558f04387d25]
addr2line -e ./t8_advection -i
0x1b256
/home/elsw_sa/git/t8code_debug/./../t8code/example/advect/t8_advection.cxx:1851
0x1963e
/home/elsw_sa/git/t8code_debug/./../t8code/example/advect/t8_advection.cxx:1371
0x18a3c
/home/elsw_sa/git/t8code_debug/./../t8code/example/advect/t8_advection.cxx:1166
0x5c539
/home/elsw_sa/git/t8code_debug/./../t8code/src/t8_forest/t8_forest_cxx.cxx:1935
0xc9981
/home/elsw_sa/git/t8code_debug/sc/.././../t8code/sc/src/sc.c:1044
0xc9760
/home/elsw_sa/git/t8code_debug/sc/.././../t8code/sc/src/sc.c:982
0xc97a5
/home/elsw_sa/git/t8code_debug/sc/.././../t8code/sc/src/sc.c:997
Davknapp commented 1 year ago

I was unable to reproduce this error. However I got some other error-messages related to pyramids, that I was able to fix. At some parts in the advection example it was assumed that the children of an element have the same number of faces as the parent, which doesn't hold for pyramids anymore.

Davknapp commented 1 year ago

I was unable to reproduce this error. However I got some other error-messages related to pyramids, that I was able to fix. At some parts in the advection example it was assumed that the children of an element have the same number of faces as the parent, which doesn't hold for pyramids anymore.

Nevermind, they are already adressed in the branch fix-issue390

sandro-elsweijer commented 9 months ago

We should check, if this PR fixed the Bug: https://github.com/DLR-AMR/t8code/pull/770

lukasdreyer commented 1 month ago

mpirun -np 8 ./t8_advection -u7 -l0 -r1 -f cube -d3 -C0.5 -b4 -X-0.7 -Y0 -Z0 -R0.3 -T1

The documentation for u7 says:

7 - Flow past a rotating cylinder of radius of 0.5 around the z-axis. This flow is defined for a specific mesh, which can be generated with Gmsh and the .geo files 't8_advection_generate_channel.geo' and 't8_advection_generate_channel_2d.geo'. These meshes can also be used with the curved geometry.

The cube mesh tries to evaluate the flow function inside of the "channel", therefore unreasonable speeds are evaluated, and the timestep cannot be chosen accurately.

With -u1, at least the volume loss gives reasonable values, although since this mesh is not periodic, the analytic solution and therefore the error estimators do not make sense. I am also not sure, how boundary conditions are handled in the advection code.

This configuration mpirun -n 32 ./t8_advection -u1 -l0 -r5 -f cube -d3 -C0.1 -b1 -X-0.5 -Y0 -Z0 -R0.2 -T0.1 gives me a volume loss of 16% in release mode and it runs in debug mode without assertions.

@sandro-elsweijer: Does this solve your problem? @Davknapp: Do you concur with my observations?