Investigate performance with affine transformation 50k point datasets

[gadomski]

We've got a report in Gitter that affine borks with 50k point datasets w/ about 32G of memory, which seems...worse than I'd expect. Though, re-reading the discussion, these are 100 dimension point sets, so maybe it's not that surprising. Check it out anyways, see if we can reproduce.

[gadomski]

Okay, so the problem looks like this:

• On the first cpd iteration, fgt, using ifgt, calculates a truncation number of 174, which is within bounds (200 is the currently-hardcoded upper limit). However, this truncation number w/ a 100-dimension dataset induces a p_max_total of 38900626766413400269643493897280213056902576233961077487817475035075152244960, which is ludicrous. The first bug is that this overflows p_max_total in fgt, which quickly creates chaos. I've opened gadomski/fgt#34 to more-gracefully handle this case upstream.
• cpd should know enough to guard against this, though exactly how I'm not quite sure. 100-column datasets isn't something I engineered for, but that doesn't mean it's something I shouldn't engineer for. At the very least, we should let the API user enable-disable IFGT so they can test performance with just direct-tree, no? Another option would be to shrink the bandwidth...an informative error message might prompt the user in that direction.

I'll add some API knobs that will make it easier to disable bits and pieces that might be getting in the way, and include them in a PR referencing this ticket to see if we can't make things a bit nicer, at least.

cc @courageon

[gadomski]

gadomski/fgt#34 is complete, meaning that we should get better (and quicker) notification if things are going awry under the hood. The second half of this ticket, the cpd part, is still unfinished — I'd like a knob to be able to turn on/off ifgt, so we can easily test to see if direct-tree is usable in this case.