347 refactor validation category

[nmaarnio]

New tools added and old ones modified. Here is a brief list of the changes:

• Removed AUC tool and related tests and docs (the reason is that AUC is calculated directly from Sklearn when a curve is calculated)
• Moved get_pa_intersection to plot_prediction_area_curves.py module, since it is needed only there
• Added module classifier_probability_evaluation.py that has the following tools:
  • summarize_probability_metrics to generate a report dictionary from probability array
  • plot_roc_curve
  • plot_det_curve
  • plot_precision_recall_curve
  • plot_calibration_curve
  • plot_predicted_probability_distribution
• Added module classifier_label_evaluation.py and the following tools
  • summarize_label_metrics_binary
  • confusion_matrix (just direct import from Sklearn, no wrapper)
• Added notebook called testing_classifier_evaluation.ipynb to demonstrate the two new modules above
• Separated scoring predictions into own module called scoring.py
• Small modification to plot_confusion_matrix
• Separated predicting with ML model functions into their own module called machine_learning_predicti.py
• Separated predict tool for classifier and regressor models, so two new functions:
  • predict_classifier, this tool now can return the probability array based on parameter. NOTE: Testing for Keras models still not adequate enough
  • predict_regressor

TODO:

• Add and update affected CLI functions
• Rename the validation folder to evaluation as the last thing

[nmaarnio]

@msmiyels , do you have time to review this PR in the next couple of days? Ended up doing quite a lot of modeling related stuff, so thought it could be good to involve you. If not, I can ask someone else

[nmaarnio]

Hey @msmiyels and thanks for the review!

I made some changes according to your review and carried out the renaming

[nmaarnio]

Do you think this could me merged @msmiyels ?

[msmiyels]

Do you think this could me merged @msmiyels ?

Hi Niko,

yep, think this PR can be merged 👍