xcompact3d / x3d2

https://xcompact3d.github.io/x3d2
BSD 3-Clause "New" or "Revised" License
3 stars 4 forks source link

Investigate removing reordered variables to reduce memory usage #41

Open JamieJQuinn opened 6 months ago

JamieJQuinn commented 6 months ago

@semi-h Can you summarise here?

semi-h commented 6 months ago

This is very briefly mentioned in #37. It only requires some changes in the solver class, so completely independent from the backends.

We reserve 3 fields for 3 components of velocity field when we instantiate the solver. https://github.com/xcompact3d/x3d2/blob/00d0986ab4501470021cfcb5fc443542cdbf4eb6/src/solver.f90#L80-L82

So long as solver lives we have 3 field size equivalent memory with us all the time. It makes sense because after all we need to do a timestep after transeq such that $u_i = u_i + dt \cdot du_i$ and then after pressure is sorted we need to do a pressure correction such that $u_i = u_i - p_x \cdot \epsilon ??$

So obviously we can't get rid of them. But inside transeq we can play a trick to limit the total amount of temporary storage transeq requires.

We first use the velocity field components here, and obtain all the derivatives we need in x direction. https://github.com/xcompact3d/x3d2/blob/00d0986ab4501470021cfcb5fc443542cdbf4eb6/src/solver.f90#L151

Then we have to reorder them into y so that y derivatives can be obtained https://github.com/xcompact3d/x3d2/blob/00d0986ab4501470021cfcb5fc443542cdbf4eb6/src/solver.f90#L162-L164

And this is the key part. Because at this stage we have a copy of velocity field in y orientation as well as the original arrays in x orientation, we can actually get rid of the memory of the original arrays.

If we do this then the reordering below will have to go from y to z. https://github.com/xcompact3d/x3d2/blob/00d0986ab4501470021cfcb5fc443542cdbf4eb6/src/solver.f90#L194-L196

And then at the very end we'll have to reorder z->x so that the original u, v, and w fields are restrored before returning back to run method in solver.