The current ici-kendallt-splitup uses some decent math to try and figure out how to split up the correlation comparisons across a large number of comparisons.
However, this frequently doesn't quite work. As in some cores still get more or fewer than ideal, and I never seem to use all the cores. I also can't depend on {future} scheduling, because the only way I can see to do that is to split all the comparisons, which will also copy things way more times than we should.
Alternatively, I should do choose(n, 2) to get all pairwise, add diagonal if needed, and then split this list of comparisons up across number of cores. This should provide better numbers of processes across cores than what I'm currently doing
rmflight
commented
4 years ago
The current ici-kendallt-splitup uses some decent math to try and figure out how to split up the correlation comparisons across a large number of comparisons.
However, this frequently doesn't quite work. As in some cores still get more or fewer than ideal, and I never seem to use all the cores. I also can't depend on {future} scheduling, because the only way I can see to do that is to split all the comparisons, which will also copy things way more times than we should.
Alternatively, I should do
choose(n, 2)to get all pairwise, add diagonal if needed, and then split this list of comparisons up across number of cores. This should provide better numbers of processes across cores than what I'm currently doing