As with all deep learning methods, sometimes training can diverge. However, our current bootstrap estimates of confidence include all training trajectories. When we crank up the number of bootstraps, we tend to include training runs that actually diverged, artificially increasing the reported uncertainty. It would be great to have an automated way to report results based only on bootstrap training runs that actually converged to a good solution.
As with all deep learning methods, sometimes training can diverge. However, our current bootstrap estimates of confidence include all training trajectories. When we crank up the number of bootstraps, we tend to include training runs that actually diverged, artificially increasing the reported uncertainty. It would be great to have an automated way to report results based only on bootstrap training runs that actually converged to a good solution.