[FEATURE] Reduce unit test duration

gramhagen commented 5 years ago

Current unit tests in notebooks are lengthy, need to investigate ways to reduce complexity or change parameters to speed up testing of notebooks.

Here are the current test durations (for any test longer than 1s) for different configurations in order to prioritize efforts.

not vw and not spark and not gpu
Duration	Test
155.85s	tests/unit/test_notebooks_python.py::test_rlrmc_quickstart_runs
41.76s	tests/unit/test_notebooks_python.py::test_lightgbm
38.63s	tests/unit/test_notebooks_python.py::test_surprise_deep_dive_runs
23.61s	tests/unit/test_notebooks_python.py::test_sar_deep_dive_runs
16.61s	tests/unit/test_notebooks_python.py::test_sar_single_node_runs
16.60s	tests/unit/test_notebooks_python.py::test_baseline_deep_dive_runs
3.51s	tests/unit/test_notebook_utils.py::test_is_jupyter
3.49s	tests/unit/test_notebooks_python.py::test_template_runs
2.00s	tests/unit/test_timer.py::test_timer

not vw and not spark and gpu
Duration	Test
155.18s	tests/unit/test_notebooks_gpu.py::test_wide_deep
125.07s	tests/unit/test_notebooks_gpu.py::test_xdeepfm
107.35s	tests/unit/test_notebooks_gpu.py::test_ncf_deep_dive
39.24s	tests/unit/test_notebooks_gpu.py::test_ncf
35.85s	tests/unit/test_notebooks_gpu.py::test_fastai
29.56s	tests/unit/test_deeprec_model.py::test_dkn_component_definition
12.80s	tests/unit/test_ncf_singlenode.py::test_fit[NeuMF]
10.34s	tests/unit/test_ncf_singlenode.py::test_predict[NeuMF]
9.77s	tests/unit/test_tf_utils.py::test_evaluation_log_hook
8.57s	tests/unit/test_ncf_singlenode.py::test_fit[MLP]
7.48s	tests/unit/test_deeprec_model.py::test_xdeepfm_component_definition
7.19s	tests/unit/test_wide_deep_utils.py::test_build_model
6.96s	tests/unit/test_ncf_singlenode.py::test_fit[GMF]
6.88s	tests/unit/test_ncf_singlenode.py::test_predict[GMF]
6.78s	tests/unit/test_ncf_singlenode.py::test_predict[MLP]
1.93s	tests/unit/test_ncf_singlenode.py::test_neumf_save_load[4-8]
1.77s	tests/unit/test_ncf_singlenode.py::test_neumf_save_load[5-5]
1.33s	tests/unit/test_deeprec_utils.py::test_DKN_iterator
1.33s	tests/unit/test_ncf_singlenode.py::test_regular_save_load[NeuMF-5-5]
1.11s	tests/unit/test_ncf_singlenode.py::test_regular_save_load[MLP-5-5]
1.07s	tests/unit/test_ncf_singlenode.py::test_regular_save_load[GMF-5-5]

not vw and spark and not gpu
Duration	Test
145.73s	tests/unit/test_notebooks_pyspark.py::test_spark_tuning
134.66s	tests/unit/test_notebooks_pyspark.py::test_als_deep_dive_runs
88.78s	tests/unit/test_notebooks_pyspark.py::test_evaluation_runs
60.56s	tests/unit/test_notebooks_pyspark.py::test_mmlspark_lightgbm_criteo_runs
51.75s	tests/unit/test_notebooks_pyspark.py::test_als_pyspark_runs
34.85s	tests/unit/test_notebooks_pyspark.py::test_data_split_runs
3.12s	tests/unit/test_spark_splitter.py::test_chrono_splitter
2.71s	tests/unit/test_spark_evaluation.py::test_spark_python_match
2.53s	tests/unit/test_spark_evaluation.py::test_init_spark
2.13s	tests/unit/test_spark_evaluation.py::test_spark_precision
1.98s	tests/unit/test_spark_evaluation.py::test_init_spark_rating_eval
1.71s	tests/unit/test_spark_splitter.py::test_timestamp_splitter
1.68s	tests/unit/test_spark_splitter.py::test_stratified_splitter
1.51s	tests/unit/test_spark_evaluation.py::test_spark_exp_var
1.49s	tests/unit/test_spark_evaluation.py::test_spark_recall
1.32s	tests/unit/test_spark_evaluation.py::test_spark_rmse
1.28s	tests/unit/test_spark_evaluation.py::test_init_spark_rating_eval
1.21s	tests/unit/test_spark_evaluation.py::test_spark_ndcg
1.20s	tests/unit/test_spark_evaluation.py::test_spark_map
1.11s	tests/unit/test_spark_evaluation.py::test_spark_mae