IndexError: list index out of range

[teng-gao]

Hi Matt,

I'm experiencing the below issue while running Cassiopeia hybrid solver on Weng et al MKP mouse dataset:

vanilla_greedy = cas.solver.VanillaGreedySolver()

hybrid_solver = cas.solver.HybridSolver(
    top_solver=vanilla_greedy, bottom_solver=ilp_solver,
    cell_cutoff=40, threads=10)

hybrid_solver.solve(cas_tree, logfile='example_hybrid.log')

---------------------------------------------------------------------------
RemoteTraceback                           Traceback (most recent call last)
RemoteTraceback: 
"""
Traceback (most recent call last):
  File "[/opt/homebrew/Caskroom/miniconda/base/lib/python3.9/multiprocessing/pool.py", line 125](http://localhost:8888/opt/homebrew/Caskroom/miniconda/base/lib/python3.9/multiprocessing/pool.py#line=124), in worker
    result = (True, func(*args, **kwds))
  File "[/opt/homebrew/Caskroom/miniconda/base/lib/python3.9/multiprocessing/pool.py", line 51](http://localhost:8888/opt/homebrew/Caskroom/miniconda/base/lib/python3.9/multiprocessing/pool.py#line=50), in starmapstar
    return list(itertools.starmap(args[0], args[1]))
  File "[/opt/homebrew/Caskroom/miniconda/base/lib/python3.9/site-packages/cassiopeia/solver/HybridSolver.py", line 355](http://localhost:8888/opt/homebrew/Caskroom/miniconda/base/lib/python3.9/site-packages/cassiopeia/solver/HybridSolver.py#line=354), in apply_bottom_solver
    self.bottom_solver.solve(subtree, logfile=logfile)
  File "[/opt/homebrew/Caskroom/miniconda/base/lib/python3.9/site-packages/ngs_tools/logging.py", line 62](http://localhost:8888/opt/homebrew/Caskroom/miniconda/base/lib/python3.9/site-packages/ngs_tools/logging.py#line=61), in inner
    return func(*args, **kwargs)
  File "[/opt/homebrew/Caskroom/miniconda/base/lib/python3.9/site-packages/cassiopeia/solver/ILPSolver.py", line 272](http://localhost:8888/opt/homebrew/Caskroom/miniconda/base/lib/python3.9/site-packages/cassiopeia/solver/ILPSolver.py#line=271), in solve
    cassiopeia_tree.collapse_unifurcations()
  File "[/opt/homebrew/Caskroom/miniconda/base/lib/python3.9/site-packages/cassiopeia/data/CassiopeiaTree.py", line 1669](http://localhost:8888/opt/homebrew/Caskroom/miniconda/base/lib/python3.9/site-packages/cassiopeia/data/CassiopeiaTree.py#line=1668), in collapse_unifurcations
    source = self.root
  File "[/opt/homebrew/Caskroom/miniconda/base/lib/python3.9/site-packages/cassiopeia/data/CassiopeiaTree.py", line 447](http://localhost:8888/opt/homebrew/Caskroom/miniconda/base/lib/python3.9/site-packages/cassiopeia/data/CassiopeiaTree.py#line=446), in root
    self.__cache["root"] = [
IndexError: list index out of range
"""

The above exception was the direct cause of the following exception:

IndexError                                Traceback (most recent call last)
Cell In[10], line 7
      1 vanilla_greedy = cas.solver.VanillaGreedySolver()
      3 hybrid_solver = cas.solver.HybridSolver(
      4     top_solver=vanilla_greedy, bottom_solver=ilp_solver,
      5     cell_cutoff=40, threads=10)
----> 7 hybrid_solver.solve(cas_tree, logfile='example_hybrid.log')

File [/opt/homebrew/Caskroom/miniconda/base/lib/python3.9/site-packages/cassiopeia/solver/HybridSolver.py:156](http://localhost:8888/opt/homebrew/Caskroom/miniconda/base/lib/python3.9/site-packages/cassiopeia/solver/HybridSolver.py#line=155), in HybridSolver.solve(self, cassiopeia_tree, layer, collapse_mutationless_edges, logfile)
    151 if self.threads > 1:
    152     with multiprocessing.Pool(processes=self.threads) as pool:
    154         results = list(
    155             tqdm(
--> 156                 pool.starmap(
    157                     self.apply_bottom_solver,
    158                     [
    159                         (
    160                             cassiopeia_tree,
    161                             subproblem[0],
    162                             subproblem[1],
    163                             None if logfile is None else 
    164                                 f"{logfile.split('.log')[0]}-"
    165                                 f"{next(logfile_names)}.log",
    166                             layer,
    167                         )
    168                         for subproblem in subproblems
    169                     ],
    170                 ),
    171                 total=len(subproblems),
    172                 disable=not self.progress_bar
    173             )
    174         )
    175 # single-threaded bottom solver approach
    176 else:
    177     results = [
    178         self.apply_bottom_solver(
    179             cassiopeia_tree,
   (...)
    188             total=len(subproblems),disable=not self.progress_bar)
    189     ]

File [/opt/homebrew/Caskroom/miniconda/base/lib/python3.9/multiprocessing/pool.py:372](http://localhost:8888/opt/homebrew/Caskroom/miniconda/base/lib/python3.9/multiprocessing/pool.py#line=371), in Pool.starmap(self, func, iterable, chunksize)
    366 def starmap(self, func, iterable, chunksize=None):
    367     '''
    368     Like `map()` method but the elements of the `iterable` are expected to
    369     be iterables as well and will be unpacked as arguments. Hence
    370     `func` and (a, b) becomes func(a, b).
    371     '''
--> 372     return self._map_async(func, iterable, starmapstar, chunksize).get()

File [/opt/homebrew/Caskroom/miniconda/base/lib/python3.9/multiprocessing/pool.py:771](http://localhost:8888/opt/homebrew/Caskroom/miniconda/base/lib/python3.9/multiprocessing/pool.py#line=770), in ApplyResult.get(self, timeout)
    769     return self._value
    770 else:
--> 771     raise self._value

IndexError: list index out of range

I'm using Python 3.9.19 and Cassiopeia 2.0.0. Also attached is the log (seems like the run finished successfully).

example_hybrid-11.log

[mattjones315]

Hi @teng-gao ,

It appears this error stemmed from the way we were assigning randomly-generated names to internal nodes during the ILPSolver.solve() method. I believe I fixed this with the latest commit c568a72 which has just been pushed into the codebase. If you pull the latest code and re-install then I believe this issue should be fixed. Please let me know if there are any other issues you come across!

Best, Matt

[teng-gao]

The new commit version works great! Thanks so much for the quick fix!