search

sandialabs / Albany

Sandia National Laboratories' Albany multiphysics code
Other
282 stars 89 forks source link

Add nightly blake intel build #1018

Closed jewatkins closed 12 months ago

jewatkins commented 12 months ago

This adds a nightly blake intel release build. It has pardiso and my first impression is that its faster than the gcc build for larger sims.

@mcarlson801 this is the first step towards a sycl build

mperego commented 12 months ago

@jewatkins At the Trilinos meeting I heard that you need to compile with -fp-model precise, otherwise you would get very different results with oneAPI compilers. Did you experience that?

jewatkins commented 12 months ago

@jewatkins At the Trilinos meeting I heard that you need to compile with -fp-model precise, otherwise you would get very different results with oneAPI compilers. Did you experience that?

I'm working that right now. All tests are passing except those that use chebyshev. I think... I guess I didn't check all the performance tests with the other solvers, I just know that linesmoothing and gauss-seidel work fine. I tried -fp-model precise and chebyshev still stalls. I'm trying something a bit more aggressive right now.

jewatkins commented 12 months ago

also pyalbany tests fail because it tries to use srun for some reason

jewatkins commented 12 months ago

It looks like -fp-model precise works for the velocity solves with chebyshev but not enthalpy.

mperego commented 12 months ago

It looks like -fp-model precise works for the velocity solves with chebyshev but not enthalpy.

Interesting. Apparently Nalu had a similar issue but in that case the -pf-model precise flag seemed to fix it. https://github.com/trilinos/Trilinos/issues/12470

jewatkins commented 12 months ago

Interesting. Apparently Nalu had a similar issue but in that case the -pf-model precise flag seemed to fix it. trilinos/Trilinos#12470

Yeah that's the issue that Jonathan pointed me to.

I'm not able to converge Enthalpy with chebyshev or a combination of MT Gauss-Seidel/Two-stage Gauss-Seidel. But we use line/plane smoothing and serial gauss-seidel on CPU so that's probably okay.

Overall -fp-model=precise seems to be faster so I think we can go ahead and use it and at least we can run with chebyshev in some cases (like the velocity solve).