dougshidong / PHiLiP

Parallel High-Order Library for PDEs through hp-adaptive Discontinuous Galerkin methods
Other
45 stars 36 forks source link

Efficient implementation with Scaling for DG Strong #219

Closed AlexanderCicchino closed 1 year ago

AlexanderCicchino commented 1 year ago

This PR addresses: 1) Remaining memory issues for high polynomial orders for linear and curvilinear meshes. 2) Provides scaling for DG strong conservative, overintegrated and entropy conserving on a straight and curvilinear mesh. Showing the entire strong DG scales at O(n^{d+1}), up to 136^3 dofs ran locally on 4GB of ram. 3) Restructuring the two-point flux and creating a vector sparsity structure as to only compute O(n^{d+1}) terms on the fly and apply them. 4) Overintegrated entropy conserving schemes now conserve entropy at machine precision on general integration strength by introducing a new solution basis entropy projected variable. Below, please find results for conservative strong DG, FR c+ with and without overintegration. Also, DG and c+ entropy conserving "EC" two-point flux form using Ranocha's two-point flux for both Cartesian and nonsymmetric curvilinear meshes. 3D_TGV_scaling_curv_legend_right.pdf 3D_scaling_cartesian_TGV_legend_right.pdf

AlexanderCicchino commented 1 year ago

For the overintegration curvilinear tests, it runs out of memory on the p=20 solve after doing the scaling. The last major memory issue for curvilinear overintegration comes from creating the MappingQGeneric object and fe value for get_position_vector in high_order_mesh. An issue after this PR should be raised to fix this by having an option to extract the volume_nodes vector from the mesh->grids->warp function. This will prevent high_order_mesh from growing in memory proportional to quadrature nodes, but instead being solely dependent on the mesh nodes. Lastly, another large memory usage comes from allocation the dim volume_quadrature_collection and fe_collection_lagrange. For DG strong, neither of these are needed so an option will need to be added where the volume quadrature weights are obtained through the tensor product of the oneD quadrature weights-- similarly for the surface quadrature weights.

AlexanderCicchino commented 1 year ago

https://github.com/dougshidong/PHiLiP/issues/221

jbrillon commented 1 year ago

I'd also like to acknowledge the size of this PR in comparison to the last one, thanks for the relatively small PR, makes the review much easier for us

AlexanderCicchino commented 1 year ago

ctest_log_Narval_18May.log The test ANISOTROPIC_MESH_ADAPTATION_SSHOCK failed because I used the wrong version of gmsh. It passes when using gmsh 4.11. 2D_SUM_FACTORIZATION_HADAMARD_TEST passes locally MPI_3D_TAYLOR_GREEN_SCALING_OVERINTEGRATION passes locally MPI_3D_TAYLOR_GREEN_SCALING_OVERINTEGRATION_CPLUS passes locally -- they failed on Narval due the cpu usage issue so the scaling is not exact at each delta(poly_degree) MPI_3D_TAYLOR_GREEN_SCALING_OVERINTEGRATION_CURV and MPI_3D_TAYLOR_GREEN_SCALING_OVERINTEGRATION_CPLUS_CURV should fail at the memory check due to issue https://github.com/dougshidong/PHiLiP/issues/221

jbrillon commented 1 year ago

can you post a log file showing that these pass? its helpful that we have an idea of the run time for the tests

AlexanderCicchino commented 1 year ago

can you post a log file showing that these pass? its helpful that we have an idea of the run time for the tests extra_ctest_results.log

jbrillon commented 1 year ago

Thanks, looks good, just add MPI_3D_TAYLOR_GREEN_SCALING_OVERINTEGRATION_CURV and MPI_3D_TAYLOR_GREEN_SCALING_OVERINTEGRATION_CPLUS_CURV as new issues with testfail

AlexanderCicchino commented 1 year ago

Thanks, looks good, just add MPI_3D_TAYLOR_GREEN_SCALING_OVERINTEGRATION_CURV and MPI_3D_TAYLOR_GREEN_SCALING_OVERINTEGRATION_CPLUS_CURV as new issues with testfail

https://github.com/dougshidong/PHiLiP/issues/223