Description of the problem
When training with TRUST_REGION, the $\chi^2$ drops down to less than $1$, resulting in strong over-learning. However, this indicates that both minimisation algorithm and implementation of the $\chi^2$ are working fine. On the other hand, GD vanilla reduces the $\chi^2$ down to $\sim 40$ even after $10000$ iterations. It is required to understand whether this is due to the very small uncertainties of the generated points or there is a bug in the implementation of the $\chi^2$. The latter is less feasible, because the AnalyticChiSquare works when using TRUST_REGION. Still, it is worth checking before drawing any conclusion.
This addresses task in #1.
Description of the problem When training with
TRUST_REGION, the $\chi^2$ drops down to less than $1$, resulting in strong over-learning. However, this indicates that both minimisation algorithm and implementation of the $\chi^2$ are working fine. On the other hand, GD vanilla reduces the $\chi^2$ down to $\sim 40$ even after $10000$ iterations. It is required to understand whether this is due to the very small uncertainties of the generated points or there is a bug in the implementation of the $\chi^2$. The latter is less feasible, because theAnalyticChiSquareworks when usingTRUST_REGION. Still, it is worth checking before drawing any conclusion.