Changes to control model sparsity and MTSS improvements

Improvements to MultitaskScaffoldSplitter:

Notable speedup of GA optimization
Added early stopping for optimization loop
Changed fitness function for Tanimoto distances to grade splits based on fraction of valid/test scaffolds with no train scaffold closer than given Tanimoto distance
Track evolution of fitness function terms during optimization to support split_diagnostics_plot function.

New module split_diagnostic_plots:

Provides single function to plot multiple aspects of split quality, given split parameters or ModelPipeline as input: Response value distributions by subset, Tanimoto distance histograms, actual vs requested split fractions, and fitness term evolution.
New individual functions to plot split fractions and fitness term evolution during GA optimization.

Sparsity-related parameters for XGBoost models:

xgb_alpha: Controls strength of L1 penalty in cost function
xgb_lambda: Controls strength of L2 penalty in cost function

New search domain parameters for hyperopt optimization of sparsity parameters:

wdp – for weight_decay_penalty
wdt – for weight_decay_penalty_type
xgba – for xgb_alpha
xgbb – for xgb_lambda (xgbl is already taken)

Feature_importance function to draw line plot of summed NN absolute feature weights vs epoch.

Hi, I have tested the following so far mainly by using various tutorial notebooks and running the functions through there. I will update this in a bit when I am finished trying out the rest of the features.

Improvements to MultitaskScaffoldSplitter:

[X] Notable speedup of GA optimization
[X] Added early stopping for optimization loop
[X] Changed fitness function for Tanimoto distances to grade splits based on fraction of valid/test scaffolds with no train scaffold closer than given Tanimoto distance
[X] Track evolution of fitness function terms during optimization to support split_diagnostics_plot function.

Comments:

definitely way faster although there's a TensorFlow deprecation warning:

WARNING:tensorflow:From [/Users/apaulson/atomsci-venv/lib/python3.9/site-packages/tensorflow/python/util/deprecation.py:588](http://localhost:8888/lab/tree/repos/AMPL_umbrella/AMPL/atomsci/ddm/examples/tutorials/atomsci-venv/lib/python3.9/site-packages/tensorflow/python/util/deprecation.py#line=587): calling function (from tensorflow.python.eager.polymorphic_function.polymorphic_function) with experimental_relax_shapes is deprecated and will be removed in a future version.
Instructions for updating:
experimental_relax_shapes is deprecated, use reduce_retracing instead

early stopping works as expected
updated grading function makes sense and the code looks fine. I had trouble following all the distance matrix calculations but the logic makes sense. (_generate_scaffold_dist_matrix, far_frac_fitness)

New module split_diagnostic_plots:

[X] Provides single function to plot multiple aspects of split quality, given split parameters or ModelPipeline as input: Response value distributions by subset, Tanimoto distance histograms, actual vs requested split fractions, and fitness term evolution.
[X] New individual functions to plot split fractions and fitness term evolution during GA optimization.

Comments:

plot_split_diagnostics() is great. I would recommend moving the tanimoto distance distribution plots to the end with the unweighted fitness scores. That way, all things that are per-task are plotted first and the user can specify num_cols=num_tasks for a neater plot.
It is confusing that axes can't be None for plot_split_fractions() and plot_fitness_terms(). You can just do a simple if axes is None: fig, ax = plt.subplots() check.

Sparsity-related parameters for XGBoost models:

[ ] xgb_alpha: Controls strength of L1 penalty in cost function
[ ] xgb_lambda: Controls strength of L2 penalty in cost function

New search domain parameters for hyperopt optimization of sparsity parameters:

[ ] wdp – for weight_decay_penalty
[ ] wdt – for weight_decay_penalty_type
[X] xgba – for xgb_alpha
[X] xgbb – for xgb_lambda (xgbl is already taken)

Feature_importance

[ ] function to draw line plot of summed NN absolute feature weights vs epoch.

ATOMScience-org / AMPL