GalaxyFinder.find_enclosed_mass can still use a ton of memory for certain simulations - Githubissues

zhafen / galaxy-dive

A general analysis package for analyzing hydrodynamic simulations of galaxy formation.

MIT License

0 stars 0 forks source link

GalaxyFinder.find_enclosed_mass can still use a ton of memory for certain simulations #9

Open zhafen opened 7 years ago

zhafen commented 7 years ago

Originally reported by Zachary Hafen (Bitbucket: zhafen, GitHub: zhafen)

Even with the low memory flag and memory_mode_divisions=100, certain snapshots were using >100GB of memory per snapshot.

Bitbucket: https://bitbucket.org/zhafen/galaxy_diver/issue/9

zhafen commented 6 years ago

Original comment by Zachary Hafen (Bitbucket: zhafen, GitHub: zhafen)

An even better option may be to use a KDTree, including a periodic boundary!

zhafen commented 6 years ago

Original comment by Zachary Hafen (Bitbucket: zhafen, GitHub: zhafen)

An alternative is to maybe use something like https://github.com/agurvich/CPotential .

zhafen commented 7 years ago

Original comment by Zachary Hafen (Bitbucket: zhafen, GitHub: zhafen)

This is all especially an issue for simulation m12m.

zhafen commented 7 years ago

Original comment by Zachary Hafen (Bitbucket: zhafen, GitHub: zhafen)

It does better when setting memory_mode_divisions to ~10, using up only ~80 GB for a single snapshot. However, it takes over an hour to do some snapshots...

Edit: Nevermind, still uses ~150 GB per snapshot.

zhafen commented 7 years ago

Original comment by Zachary Hafen (Bitbucket: zhafen, GitHub: zhafen)

Changing memory_mode_divisions to something ridiculous like 1000 doesn't seem to help this.