mchaput / whoosh

Pure-Python full-text search library
Other
580 stars 73 forks source link

multiprocessing writer without multisegment raises exception #35

Open ajslater opened 1 year ago

ajslater commented 1 year ago

If I write an index with procs > 1 and multisegment = False, it raises the following error:

Process SubWriterTask-1:2:3:
Traceback (most recent call last):
  File "/opt/homebrew/Cellar/python@3.10/3.10.9/Frameworks/Python.framework/Versions/3.10/lib/python3.10/multiprocessing/process.py", line 314, in _bootstrap
    self.run()
  File "/Users/aj/Code/codex/.venv/lib/python3.10/site-packages/whoosh/multiproc.py", line 129, in run
    runname, fieldnames, segment = finish_subsegment(writer, k)
  File "/Users/aj/Code/codex/.venv/lib/python3.10/site-packages/whoosh/multiproc.py", line 49, in finish_subsegment
    runname = writer.pool.runs[0]
IndexError: list index out of range

My workaround is to always use multisegment if I set procs > 1.