Builds on #674 to reduce conflicts although the changes should be effectively orthogonal.
Uses process based parallelism to accelerate the search index generation. Tests of thread based parallelism didn't seem to show much gain but it should be easy to switch from process to thread here.
We may also want to consider adding lxml as a dependency -- this can be installed from pypi and seems to give a decent speedup to the beautiful soup parsing.
Builds on #674 to reduce conflicts although the changes should be effectively orthogonal.
Uses process based parallelism to accelerate the search index generation. Tests of thread based parallelism didn't seem to show much gain but it should be easy to switch from process to thread here.