Closed shawnlaffan closed 10 months ago
Experiments with the North American data set from Mishler et al. (2020) showed very few groups had the same SHA sum. While smaller data sets might have similar overlaps, there will be an even lower overlap for those that use sample counts. This adds complexity to the implementation for what might be only marginal gains in speed.
Marking as a wontfix.
A possible optimisation for calculation of pairwise matrices using turnover indices is to generate the SHA digest for the label sets in each group.
Any groups with the same SHA digest will contain the same labels and therefore have zero turnover.
We can also track the pairs of SHA sums to detect cases where the same turnover score will be obtained.