keras model - CV

[germayneng]

Hi CPMP,

I have read your solutions explanation at https://www.kaggle.com/c/web-traffic-time-series-forecasting/discussion/39395 and would like to say thank you for giving the solution.

I do have one question and hope you can clarify for me. Your final model is an xgb model using keras as well as your GLM's predictor as features along with some features you did.

But in order use the keras and GLM model, you would require OOF predictions for September 13th, 2017 and November 13th, 2017. (which you did output)

For the keras model, to generate the OOF, you need to perform some form of CV, and you did explain that you did via group Kfold. My question would be when doing so, there will be some folds where you will train the future to predict on the past. Is this aspect alright?

[jfpuget]

Hi,   I don't do time based CV in that case as explained in my write up.  Have a look at the code in case of doubt.  I define the folds by splitting the pages, not by splitting the dates. Best regards/Cordialement,JFJean-Francois Puget, PhDDistinguished Engineer, Machine Learning and Optimization, IBM AnalyticsMobile: +33(0)6 7000 8815 Email: j-f.puget@fr.ibm.comBlog: http://bit.ly/jfpuget Twitter: @JFPuget   Admin: Lysiane Huou +33-240414410 (Office) Lysiane.Huou1@fr.ibm.com     ----- Original message -----From: Germayne notifications@github.comTo: jfpuget/Kaggle Kaggle@noreply.github.comCc: Subscribed subscribed@noreply.github.comSubject: [jfpuget/Kaggle] keras model - CV (#3)Date: Wed, Jun 27, 2018 4:43 PM  Hi CPMP, I have read your solutions explanation at https://www.kaggle.com/c/web-traffic-time-series-forecasting/discussion/39395 and would like to say thank you for giving the solution. I do have one question and hope you can clarify for me. Your final model is an xgb model using keras as well as your GLM's predictor as features along with some features you did. But in order use the keras and GLM model, you would require OOF predictions for September 13th, 2017 and November 13th, 2017. (which you did output) For the keras model, to generate the OOF, you need to perform some form of CV, and you did explain that you did via group Kfold. My question would be when doing so, there will be some folds where you will train the future to predict on the past. Is this aspect alright? —You are receiving this because you are subscribed to this thread.Reply to this email directly, view it on GitHub, or mute the thread.  

[germayneng]

Hi ,

I re-read and understood. It seems you are using feed forward NN, which to my understand, allows you to output multiple target. Each target are the dates (w0d1,w0d2....) and each row of your train data are unique pages. So when you define your folds via pages, the inputs which you built using the past will predict multiple output (w0d1...)

is my understand correct?

Regards Germayne

[germayneng]

Hi CPMP,

1) for the final xgb ensemble, you did 5 fold cv split. In this case, wont there be case where some of the folds are trained on the 'future' data and predicted on the past data?

Essentially each rows are different page views of a particular day in 2016. Is it in this case, you decided to not treat each row as a time ordered data but just as individual samples in this ensembling step ( since you are training on residual as targets)

2) notice that you fitted back your training data into the model that you trained with for your GLM. what is the story behind this? I understand this is the OOF, but why fit it back to the original training data ( after u done it, then u proceed to add .ALLvisits)