Open ttimbers opened 1 month ago
Few more important potential threats can be as follows:
Poor hyperparameter tuning (selecting wrong learning rate, too small or too high gamma value in SVM, depth of tree in decision trees or number of n-estimators in Random Forest)
Skewed Classes in training set ( This can lead too much training for majority class and less training on minority class)
Model Scalability ( Increase in prediction latency when then inflow velocity is high )
Thank-you for these suggestions @H234J!
I am not sure if this is part of the reproducibility issue, or should be separated:
Previously, I encountered situations like a project failed to run, or, the model produced different outputs following upgrades of underlying dependencies.
another potential mistake:
Here I want to brainstorm a list to what are all the potential threats (i.e., where can things go wrong) to a machine learning project? Our checklist need not address all of them, but we should in our literature review describe them all, and identify which our checklist covers. Here's my starting list: