Open liangchen1ceeee opened 3 years ago
Thanks for pointing out the issue. The text matched the results at the time of writing the colab, but due to changes in the training for GBT or DNN models the results have moved around a bit. The primary reason for this instability is that we cannot train the models until convergence as the tutorials are auto generated from the colabs and need to run in a few minutes.
We are updating the colab to use less data and a more aggressive learning rate to get closer to convergence, re-optimizing the hyper parameters along the way. These changes will be reflected in the upcoming release.
Thanks. Look forward to your new results!
Hi, I'm looking at the shape constraints tutorial, and the results for GBDT and DNN are listed in the tutorial as follows:
After the experiment results, the tutorial comments
Note that even though the validation metric is better than the tree solution, the testing metric is much worse.
I don't understand where this comment comes from, since DNN outperforms GBT in both validation AUC and testing AUC.