As noted in the "Potential future work" section of PR #681:
The threading performance of annif hyperopt was already bad, and now it got worse. The solution could be to switch to process-based multiprocessing. This has been difficult to do with Optuna (needs an external relational database), but the Optuna FAQ now states that it could also be done with JournalFileStorage, which sounds more promising.
So we should investigate whether it would be possible to use multiprocessing in hyperopt operations, because the current multithreading approach doesn't actually work very well.
As noted in the "Potential future work" section of PR #681:
So we should investigate whether it would be possible to use multiprocessing in hyperopt operations, because the current multithreading approach doesn't actually work very well.