Currently, the kmers index generation routine ("precalc step") utilities concurrent threading. Use cases (such as the WG dataset) have shown that single node concurrent threading is insufficient.
Parallel processing should occur across several nodes. There is no need to communicate between nodes. The current workaround involves several independent instances of gramtools. The downside of this workaround in it's current state is that each instance of gramtools must have a unique copy of the "gram files" directory, otherwise IO race conditions can occur.
Currently, the kmers index generation routine ("precalc step") utilities concurrent threading. Use cases (such as the WG dataset) have shown that single node concurrent threading is insufficient.
Parallel processing should occur across several nodes. There is no need to communicate between nodes. The current workaround involves several independent instances of gramtools. The downside of this workaround in it's current state is that each instance of gramtools must have a unique copy of the "gram files" directory, otherwise IO race conditions can occur.