Open sjordan29 opened 9 months ago
@sjordan29, this is great stuff. Thanks for finding and figuring out how to use memory_profiler!
I created this complementary issue that documents some of the approaches we discussed this week.
A few key findings from the commit above:
coeff_to_diffusion
has the largest memory of all the variables, even though it has the same dimensions ('time', 'nedge') as the others. this is because it's a float64
whereas the others are float32
(I think because they are read directly from the HDF file and that must be the precision of the RAS output). sel
data from the xarray in memory than from a lazy-loaded xarray (when loading the values).
I used
memory_profiler
for instantiation of Clearwater-riverine and running a few timesteps. The following were interesting findings:calculate_required_parameters
method is very memory intensive, because transport calculations require a few key variables with dimensions oftime
by thenumber of edges
, which get very large as the time dimension grows.clearwater-modules
helped with memory managementNext steps:
FLOW_ACROSS_FACE
variable via #60 to reduce memory-intensive redundancy