MS (takazawa) - Try first simple model

[yukiregista]

Implement the first simple model using some natural encoding.

• Target-encode some categorical variables with a large number of categories.
• Use LightGBM
• [ ] Upload the code and record submission performance
• [ ] Make data-cleaning function

[rkondo3]

I have finished the 1st item on branch:feature/kondo/init. But I've never submitted it.

[yukiregista]

I have uploaded two notebooks (firstmodel.ipynb and firstmodel_with_threshold.ipynb) which do very simple LightGBM modeling with some natural encodings of categorical features. firstmodel.ipynb (predicting 1 if output > 0.5) scored around 0.65 in the leaderboard. firstmodel_with_threshold (threshold of output decided depending on the performance) scored around 0.68 in the leaderboard (which is not too bad considering no hyperparameter optimization or fancy feature engineering is done).

[yukiregista]

To do and ideas for improvement

• [ ] Organize the codes ( esp. make functions for some data-handling, evaluation, and model building parts so that we can use it easily later)
• [ ] Improve on how threshold is decided; This inclues
  • [ ] Larger range for the search of threshold (currently 100 grids from 0.2 to 0.8, but 0.8 is adopted)
  • [ ] Decide threshold for each cv's booster based on the evaluation on the validation set of that fold
• [ ] Make more features; Do EDA part 2, where we generate a lot of different features and measure its relevance to the objective.
• [ ] Hyperparameter tuning (optuna)
• [ ] Different loss function (?)
• [ ] Class weights (?)
• [ ] Encoding City as well (currently City is ignored); we could use similar target encoding but with smoothing based on distance on sphere (earth)