Enable caching capability in Material

[YaqiWang]

Reason

We have cases where material property evaluation is costly but they are not changing during linear iterations. So we want to cache the properties on all quadrature points possibly during Jacobian evaluation and later reuse them during residual evaluations. More specifically, the evaluation involves a large table look-up/interpolation and mixing of hundreds of isotopes. The cost of evaluation is probably much higher than retrieving the cached values in the memory. Caching will increase the memory consumption, but it is still worthwhile considering the gain in CPU-time.

Design

A conversation from @rwcarlsen can be useful for tackling this issue:

I looked into using stateful properties a while back to do (current) material prop caching. The modifications actually >aren't too difficult. Then you'd just have an invalidateCache function that we call based on execute_on flags or >otherwise as desired that resets/erases the cached props so they are recomputed next time computeProperties is called. If I remember right it wasn't that big of a change, but it was pre-covid - so a while ago.

@rwcarlsen mentioned that

I could maybe put up a PR of my old/original work though for you to look at.

It is also doable for caching a selected properties in a material but not all of them if desired. This feature is what @jortensi want in some of his models. He and @permcody can decide how we approach this once a PR is up by @rwcarlsen. @vincentlaboure could be also interested in this.

Impact

A new capability that can have large impact for certain types of simulations.

[rwcarlsen]

Apparently - I did open a PR way back - it was closed as stale: https://github.com/idaholab/moose/pull/13848

[lindsayad]

These properties change from nonlinear to nonlinear? Why are the only constant during linear?

[GiudGiud]

I think some properties are dependent on buckling for some cases, which is computed from the non-linear variables.

[lindsayad]

So if we are doing call-backs during the linear solve, which indicates to me that we are doing JFNK, in theory those properties that are dependent on the nonlinear variables probably should be re-computed. But I guess we don't care if our approximated Jacobian action loses some accuracy?

[GiudGiud]

not sure what you mean by call-backs here but yes, we care more about the cost of evaluating those XS than the loss of accuracy in the Jacobian action

[vincentlaboure]

In most cases, those material properties (XS) will only have to be updated when the transfers are performed (so usually once per Picard iteration, in which case evaluating on nonlinear will also be unnecessary)

[lindsayad]

By call-back I mean calls back to MOOSE during PETSc's linear solve. When doing Newton, there are not any call-backs during the linear solve, but JFNK there are the call-backs (e.g. residual evaluations) in order to determine the approximation of the Jacobian action on a vector

[YaqiWang]

Even material properties do change on linear, making them evaluated on nonlinear only will essentially lag them. We may require more nonlinear iterations but the overall cost can possibly still be reduced.