ValueError: too many values to unpack (expected 7)

[mikeyupic]

Hi,

I'm trying to run dynesty on a Ray cluster. I encountered the error below when I ran the Examples -- Exponential Wave:

#from multiprocessing import Pool
from ray.util.multiprocessing import Pool
with Pool() as pool:
    sampler = dynesty.NestedSampler(loglike, prior_transform, 7, 
                                    periodic=[2, 5],
                                    sample='slice', nlive=250, pool=pool, queue_size=16)
    sampler.run_nested(dlogz=0.01)
    res = sampler.results

2020-12-04 13:00:11,646 INFO services.py:1090 -- View the Ray dashboard at http://127.0.0.1:8267
0it [00:00, ?it/s]
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-6-6c3892e2997a> in <module>
      6                                     periodic=[2, 5],
      7                                     sample='slice', nlive=250, pool=pool, queue_size=16)
----> 8     sampler.run_nested(dlogz=0.01)
      9     res = sampler.results

~/anaconda3/lib/python3.8/site-packages/dynesty/sampler.py in run_nested(self, maxiter, maxcall, dlogz, logl_max, n_effective, add_live, print_progress, print_func, save_bounds)
    919         try:
    920             ncall = self.ncall
--> 921             for it, results in enumerate(self.sample(maxiter=maxiter,
    922                                                      maxcall=maxcall,
    923                                                      dlogz=dlogz,

~/anaconda3/lib/python3.8/site-packages/dynesty/sampler.py in sample(self, maxiter, maxcall, dlogz, logl_max, n_effective, add_live, save_bounds, save_samples)
    780             # `logl > loglstar` using the bounding distribution and sampling
    781             # method from our sampler.
--> 782             u, v, logl, nc = self._new_point(loglstar_new, logvol)
    783             ncall += nc
    784             self.ncall += nc

~/anaconda3/lib/python3.8/site-packages/dynesty/sampler.py in _new_point(self, loglstar, logvol)
    378         while True:
    379             # Get the next point from the queue
--> 380             u, v, logl, nc, blob = self._get_point_value(loglstar)
    381             ncall += nc
    382 

~/anaconda3/lib/python3.8/site-packages/dynesty/sampler.py in _get_point_value(self, loglstar)
    362         # If the queue is empty, refill it.
    363         if self.nqueue <= 0:
--> 364             self._fill_queue(loglstar)
    365 
    366         # Grab the earliest entry.

~/anaconda3/lib/python3.8/site-packages/dynesty/sampler.py in _fill_queue(self, loglstar)
    351         if self.use_pool_evolve:
    352             # Use the pool to propose ("evolve") a new live point.
--> 353             self.queue = list(self.M(evolve_point, args))
    354         else:
    355             # Propose ("evolve") a new live point using the default `map`

~/anaconda3/lib/python3.8/site-packages/ray/util/multiprocessing/pool.py in map(self, func, iterable, chunksize)
    541         """
    542 
--> 543         return self._map_async(
    544             func, iterable, chunksize=chunksize, unpack_args=False).get()
    545 

~/anaconda3/lib/python3.8/site-packages/ray/util/multiprocessing/pool.py in get(self, timeout)
    144             for result in batch:
    145                 if isinstance(result, PoolTaskError):
--> 146                     raise result.underlying
    147                 elif isinstance(result, Exception):
    148                     raise result

ValueError: too many values to unpack (expected 7)

Is there a way to fix this? Thanks a lot!

[joshspeagle]

Looks like this is an issue with how the pool.map function is invoked here. dynesty assumes that map and pool.map can be swapped out (this is aliased via self.M) and so they both return similar iterables, but it looks like that isn't properly happening in this case.

[joshspeagle]

Closing this for now.