Try out more models / model architectures in exploration phase

[DJCordhose]

We only use a single, pretty oversized deep learning model. Especially after the first iteration we see this overfit, so it might be a good continuation of the story to use something else that does not overfit in the next iteration.

Such a model could be

• smaller neural network
• decision tree / random forest (sklearn)
• boosted tree (https://xgboost.readthedocs.io/en/stable/)
• tabular transformer (https://towardsdatascience.com/transformers-for-tabular-data-b3e196fab6f4)
• many more...

[DJCordhose]

More specifically, these would be the algorithms to try first

• [x] https://scikit-learn.org/stable/modules/tree.html#classification (easy)
• [ ] https://scikit-learn.org/stable/modules/neighbors.html#nearest-neighbors-classification (easy): #10
• [x] https://scikit-learn.org/stable/modules/ensemble.html (easy / medium)
• [ ] https://xgboost.readthedocs.io/en/stable/python/python_intro.html (medium): #9

It would be easiest to work on Colab where most of the software is already installed and you do not have to deal with a local installation. A good starting point would be the existing implementation using TensorFlow:

https://colab.research.google.com/github/openknowledge/mlops-data2day/blob/main/notebooks/exploration.ipynb

It makes sense to make a copy of this notebook and just change the parts where the architecture is created and trained on the data.

[DJCordhose]

There is a streamlined version of the existing TensorFlow implementation that runs fine on Colab and should be a better starting point

https://colab.research.google.com/github/openknowledge/mlops/blob/main/notebooks/exploration-tensorflow.ipynb

Everything before this cell

https://colab.research.google.com/github/openknowledge/mlops/blob/main/notebooks/exploration-tensorflow.ipynb#scrollTo=uN0ZEjq0nhYh

should be universal to all approaches. So new approaches would replace those parts after "ML Approach: Neural Network"