search

CEED / libCEED

CEED Library: Code for Efficient Extensible Discretizations
https://libceed.org
BSD 2-Clause "Simplified" License
187 stars 45 forks source link

Memory efficiency when assembling diagonal of a linear operator #1618

Closed sebastiangrimberg closed 2 days ago

sebastiangrimberg commented 3 days ago

If I understand the code correctly, if I assemble the diagonal using CeedOperatorLinearAssembleAddDiagonal, the assembled QFunction is kept in memory as part of the CeedQFunctionAssemblyData and not destroyed until the operator is destroyed. If I don't plan on assembling the QFunction for the operator again, it might be nice to be able to release this memory. I think I have observed this being a problem when assembling diagonals for several different operators and keeping the operators around as well.

First, is my understanding correct, and second, is there a recommended way to free this memory? Should I just call CeedQFunctionAssemblyDataDestroy of op->qf_assembled? I suppose I could also call CeedOperatorAssemblyDataDestroy on op->op_assembled, though I think this data structure is not as large in memory. Thanks for your help!

jedbrown commented 3 days ago

I think it would make sense to have a CeedOperator method to free any data that can be rebuilt. Verb could be Compress or Strip?

This could be a flag to eagerly destroy temporary data structures (rather than persist them, which is the default), though if you assemble the diagonal and then assemble a point-block diagonal (or do full assembly), the inner assembly would be done twice.

sebastiangrimberg commented 3 days ago

That sounds like a good idea. I think a good place to start is just adding the method for the user to call and having a flag can be a later step. In general I would expect the user should be aware of the implications of destroying the temporary data structures causing duplicated work if they need them to be recreated later on. One place this might not be so clear is CeedOperatorMultigridLevelCreate, where if you were to free the assembled QFunction data before the call you would assemble the QFunction twice when getting diagonals for each multigrid level, while you could avoid this if waiting until later to free the temporary data. But again, I think that's fine to put on the user to discern and the default behavior will be to keep everything around in memory as it is now.

sebastiangrimberg commented 2 days ago

See #1619. Open to renaming or any other adjustments, not sure if there is other temporary data I missed.