search

yt-project / yt

Main yt repository
http://yt-project.org
Other
466 stars 276 forks source link

AMRGridPatch objects take up too much memory #4685

Open matthewturk opened 1 year ago

matthewturk commented 1 year ago

Long ago, we optimized the AMRGridPatch objects for memory size. As a result of the gradual drift in focus, we've stopped using __slots__, among other things.

For the Santa Fe Lightcone dataset (RD0036), the grids (~400k) take up about 3.4 gb of data, which translates to 8.5kb per object. This is not that much per grid! But it's still a lot, when you add it all up.

We can either attempt to optimize these grids, which would be feasible but probably annoying, or we could move to the grid tree wholesale.

neutrinoceros commented 1 year ago

Why did we stop using __slots__ and should we consider being it back ?

matthewturk commented 1 year ago

I am not entirely sure. I think we stopped because we had a nested class hierarchy that made it hard to work with.