MKL/Lapack Update

[suzanmanasreh]

Additions

• [X] Wall force vectors
• [X] Distance from wall function
  • Helpful for analysis but could be removed because it's not used anywhere besides personal test cases (?)
• [X] Link lapack lib and use it in ModBasicMath.F90 instead of lapack95
  • [X] Matrix_PseudoInvert + QuadFit_1D correctly perform their functions with DPOSV (checked via local test cases)
  • [X] QuadFit_2D produces the same coefficients inside the simulation with DPOSV and returns no error
• [X] Link new MKL from cluster modules
  • Not sure if Intel MKL 2020 link options are correct, but it compiles. I generated them from here
• [X] Create lib directory for petsc tar file because wget old petsc …… from dropbox doesn’t work, and it makes it easier to make a commit if I can leave the packages directory in the .gitignore
  • Should be able to remove lib directory completely in next PR with petsc update
• [X] Fix carotid_web example case compilation error
• [X] Better case readme
• [X] Minor formatting changes

Problems:

• Build updates haven’t been thoroughly tested on anyone’s build besides mine. I was hoping to make a build script for phoenix in the future though if the new petsc works

[sbryngelson]

So we still need a very old PETSc or no?

[sbryngelson]

Can you verify this works on other clusters, like Rogues Gallery / ICE / Phoenix.

[sbryngelson]

Is the user out of luck if they don't have access to the correct version of MKL, or MKL is not available as a module? Many clusters don't have an MKL module.

[suzanmanasreh]

Yes, we're still using the old petsc here, so the second task on the ticket is still open. I just thought it would be too many changes for 1 PR.

This works on Phoenix and ICE because they both have mkl modules, but the rogues gallery is still impossible because the old petsc depends on python2. I think a user should be able to download any new version of mkl and use it with the Makefile, but I can include instructions on how to get one online?

[sbryngelson]

Agreed. And I think it would be useful to do that (include instructions on how to deal with MKL if it's unavailable on the user's system).

[suzanmanasreh]

I included links to instructions that would allow someone to download the same intel oneapi library that's available on the cluster, but let me know if it needs more detail.

A better solution would probably be to get rid of mkl entirely by linking lapack to blas instead, but I wanted to keep mkl because it's easy to pull the lapack and blas libraries from it for the new petsc configuration. I'm not sure if it's better to go for cluster modules or manually installed libraries.