Open wkdavis opened 2 years ago
Literature: Not sure why they didn’t check assumptions, but when I was reading through some of the papers I got the sense that they weren’t very high quality.
Feature Engineering: Deal with outliers such as Humidity = 0 values that aren’t realistic
Modeling: I’d like to get some more experience with deep learning methods and given the nature of the dataset I agree that some kind of RNN would be a good choice.
Evaluation: Just to make sure I understand the time series cv that you propose, does this look correct?
Thanks!
Literature: agreed. I think we can spend some time in the introduction poking holes in the existing literature. Then in the modeling piece we can talk about how/why our chosen methods are better.
Modeling: Awesome! Feel free to take the ML piece and run with it.
Evaluation: that's correct. I think in our case we can use a step-size of 24 observations (hours), so basically everyday at 8pm forecast the next 27 hours. Once we have some modeling ideas/code I'll generate the training and test sets for us and we'll run it just like this example: https://otexts.com/fpp3/tscv.html
Literature: We should access the model assumptions for sure.
Feature Engineering: Miss data Imputation: https://cran.r-project.org/web/packages/imputeTS/vignettes/imputeTS-Time-Series-Missing-Value-Imputation-in-R.pdf Humidity = 0 is not realistic for sure. However, I was searching for similar projects that people have done. I found some people use data from Washington D.C. and the minimum value of humidity is also 0. We can talk more during the meeting.
Modeling: I'd like to add the SARIMA model to time series methods, which is adding seasonal components to ARIMA (Autoregressive Integrated Moving Average) model. I can do ML - LSTM and other algorithms, too.
I like the imputeTS approach!
The R implementation of ARIMA in the fable package automatically checks for seasonality/seasonal ARIMA.
I will work on some code to create the CV folds for model training.
@PeiYinY Boruta feature selection: https://towardsdatascience.com/boruta-explained-the-way-i-wish-someone-explained-it-to-me-4489d70e154a
Modeling:
Imputation:
Other: