This removes the lru_cache decorators on some line-of-sight methods. I'm trying to fix some out of memory errors while generating a training set on Sherlock, and while I think #41 is the main problem, these could also contribute.
First, these caches don't actually provide a measurable speedup, at least not when generating a small dataset on my laptop.
Second, lru_cache decorators are tricky when used on methods , see e.g. here or here.
The self argument is included in the caching calculation, but does not change its hash when its attributes are updated. For example, if we were to marginalize over the cosmology, the answers of some of these functions should change as the self.cosmo attribute changes -- but they won't because they are cached.
The cache is kept globally with the class, not with a class instance. That's probably OK here since each ConfigHandler only has one LOS instance, but it is going to cause headache if we're ever going to parallelize with multiple ConfigHandlers.
Finally, the power_law_dn_dm cache decorator has an an unbound size. When redshifts are varying, this should cause a (probably small) memory leak. (Besides making the cache of any function that takes a redshift argument useless -- at least as regards to reuse of redshift-dependent computations between images)
Coverage decreased (-0.008%) to 95.035% when pulling 6f1c419a3e5b8a193da615cdd2d77058f9110c80 on no_lru_cache into 2edd7f418a63273d5b2fcc75819e811bceb1f149 on main.
This removes the
lru_cache
decorators on some line-of-sight methods. I'm trying to fix some out of memory errors while generating a training set on Sherlock, and while I think #41 is the main problem, these could also contribute.First, these caches don't actually provide a measurable speedup, at least not when generating a small dataset on my laptop.
Second,
lru_cache
decorators are tricky when used on methods , see e.g. here or here.self
argument is included in the caching calculation, but does not change its hash when its attributes are updated. For example, if we were to marginalize over the cosmology, the answers of some of these functions should change as the self.cosmo attribute changes -- but they won't because they are cached.Finally, the
power_law_dn_dm
cache decorator has an an unbound size. When redshifts are varying, this should cause a (probably small) memory leak. (Besides making the cache of any function that takes a redshift argument useless -- at least as regards to reuse of redshift-dependent computations between images)