Currently, the version of automatic calibration that has performed the best is from before we fixed the bug that artificially reduced the number of fragments we picked relative to our intention; it always picked the 500 best fragments. It is possible that calibrating on this higher-quality set meaningfully improves performance and this can be tested by making this parameter configurable.
NB: I have not made the min_fragments configurable because I don't think that the min_fragments code actually has any effect on the way the code currently runs (if there are fewer fragments than min_fragments, calibration still proceeds with the lower number). I may therefore remove it soon.
Currently, the version of automatic calibration that has performed the best is from before we fixed the bug that artificially reduced the number of fragments we picked relative to our intention; it always picked the 500 best fragments. It is possible that calibrating on this higher-quality set meaningfully improves performance and this can be tested by making this parameter configurable.
NB: I have not made the min_fragments configurable because I don't think that the min_fragments code actually has any effect on the way the code currently runs (if there are fewer fragments than min_fragments, calibration still proceeds with the lower number). I may therefore remove it soon.