ludwig-ai / ludwig

Low-code framework for building custom LLMs, neural networks, and other AI models
http://ludwig.ai
Apache License 2.0
10.97k stars 1.18k forks source link

Twitter Bots Example Overfits "Out-of-the-Box" #4012

Open Quinn-Meyer-Sustainment opened 1 month ago

Quinn-Meyer-Sustainment commented 1 month ago

I am running the Google Colab example of Ludwig-AI Multimodal Classification from the Twitter Bots Kaggle dataset. You can get the Python API Colab link directly from Ludwig AI's documentation here.

I do not edit anything in the Colab notebook at all, I simply 'Run All Code Cells' and my model over fits on the training data.

You can find my training results here: training_results.txt

And evaluation images here: training_loss confusion_matrix

Colab is running Python 3.10.12 and the pip freeze can be found here: pip_freeze.txt

Do I need to tune parameters under the hood, or should it simply be working out of the box?

Quinn-Meyer-Sustainment commented 1 month ago

Forgot to mention that the example clearly shows the model training on Kaggle data converging quite nicely... See the evaluation images here.

[Edited for semantics]