mmaninetti
commented
1 month ago Dear Dr. Snow,
Thanks for your comment! The thesis project is now finished. If you are interested, I can share a copy of the thesis with you :)
Regarding the other methods to be tested, I agree that this is some interesting future work, and thanks for the suggestions.
Regarding your second question:
- the model I am using is the standard lightgbm model, in the appendix of the thesis you can find more information on the hyperparameters' search space.
- I think it is not so surprising that lightgbm is the best performer, because other methods (especially NNs) can have some crazy behaviour outside of the training domain, while lightgbm will be more conservative in its prediction. So maybe, instead of performing "better", it is just performing "less worse".
- Additionally, defining the extrapolation set in a multidimensional setting is a task in itself. We tested different techniques to do so here, but I think that ideally in future work it should be possible to identify the most adequate technique. Anyway, I think that multidimensional real world settings are still very different from a one dimensional setting, when defining what is extrapolation is very immediate.
- To understand if data are nonstationary, I don't think that it is a very good idea to compare lightgbm results VS results of other methods. I would rather divide the data using a random train-test split, and our techniques to define the extrapolation set, and check that the methods attain a smaller error in the first case.
- We used the real world datasets used in https://arxiv.org/abs/2207.08815 . It is very likely that some of these are stationary. However, we think they are a good sample of real world datasets, and the different levels of extrapolation present in them
I hope this helped, and thanks again for the suggestions! :)
Hi, thats a cool thesis idea. You probably want to look at LightGBM with linear weak learner and KTBoost, SnapBoost, PolyBoost as well.
Just looking at your prelimenary results, is the gradient boosting model you are using just the standard of the shelf model, it seems to be doing suprisingly well, if it is doing well, does it mean that the dataset you are using are not representative of nonstatationary data for which we expect to see the benefit of extrapolative models?
Thanks in advance.