[FEATURE] Improve test coverage

[miguelgfierro]

Description

conda activate reco_full
pip install pytest-cov
pytest tests/unit/ --cov=reco_utils/ --disable-warnings

---------- coverage: platform linux, python 3.6.10-final-0 -----------
Name                                                                        Stmts   Miss  Cover
-----------------------------------------------------------------------------------------------
reco_utils/__init__.py                                                         10      0   100%
reco_utils/azureml/__init__.py                                                  0      0   100%
reco_utils/azureml/aks_utils.py                                                17      0   100%
reco_utils/azureml/azureml_utils.py                                            21     21     0%
reco_utils/azureml/svd_training.py                                             81     81     0%
reco_utils/azureml/wide_deep_training.py                                       73     73     0%
reco_utils/common/__init__.py                                                   0      0   100%
reco_utils/common/constants.py                                                 10      0   100%
reco_utils/common/general_utils.py                                             13      4    69%
reco_utils/common/gpu_utils.py                                                 67     19    72%
reco_utils/common/notebook_memory_management.py                                50     50     0%
reco_utils/common/notebook_utils.py                                            16      4    75%
reco_utils/common/plot.py                                                      34      1    97%
reco_utils/common/python_utils.py                                              40      2    95%
reco_utils/common/spark_utils.py                                               25      6    76%
reco_utils/common/tf_utils.py                                                 128     10    92%
reco_utils/common/timer.py                                                     28      0   100%
reco_utils/dataset/__init__.py                                                  0      0   100%
reco_utils/dataset/amazon_reviews.py                                          354    309    13%
reco_utils/dataset/blob_utils.py                                                6      3    50%
reco_utils/dataset/cosmos_cli.py                                               31     31     0%
reco_utils/dataset/covid_utils.py                                              47     21    55%
reco_utils/dataset/criteo.py                                                   53     25    53%
reco_utils/dataset/download_utils.py                                           44      5    89%
reco_utils/dataset/mind.py                                                    190    190     0%
reco_utils/dataset/movielens.py                                               178     36    80%
reco_utils/dataset/pandas_df_utils.py                                         134     17    87%
reco_utils/dataset/python_splitters.py                                         57      5    91%
reco_utils/dataset/spark_splitters.py                                          72      6    92%
reco_utils/dataset/sparse.py                                                   60      5    92%
reco_utils/dataset/split_utils.py                                              48      7    85%
reco_utils/dataset/wikidata.py                                                 71     11    85%
reco_utils/evaluation/__init__.py                                               0      0   100%
reco_utils/evaluation/python_evaluation.py                                     99      6    94%
reco_utils/evaluation/spark_evaluation.py                                     113     24    79%
reco_utils/recommender/__init__.py                                              0      0   100%
reco_utils/recommender/cornac/__init__.py                                       0      0   100%
reco_utils/recommender/cornac/cornac_utils.py                                  23      0   100%
reco_utils/recommender/deeprec/DataModel/ImplicitCF.py                         87     16    82%
reco_utils/recommender/deeprec/__init__.py                                      0      0   100%
reco_utils/recommender/deeprec/deeprec_utils.py                               169     49    71%
reco_utils/recommender/deeprec/io/__init__.py                                   0      0   100%
reco_utils/recommender/deeprec/io/dkn_iterator.py                             153     42    73%
reco_utils/recommender/deeprec/io/iterator.py                                 105      7    93%
reco_utils/recommender/deeprec/io/nextitnet_iterator.py                       123    110    11%
reco_utils/recommender/deeprec/io/sequential_iterator.py                      227    210     7%
reco_utils/recommender/deeprec/models/__init__.py                               0      0   100%
reco_utils/recommender/deeprec/models/base_model.py                           313    109    65%
reco_utils/recommender/deeprec/models/dkn.py                                  177     20    89%
reco_utils/recommender/deeprec/models/graphrec/lightgcn.py                    163     77    53%
reco_utils/recommender/deeprec/models/sequential/nextitnet.py                  61     51    16%
reco_utils/recommender/deeprec/models/sequential/rnn_cell_implement.py        282    254    10%
reco_utils/recommender/deeprec/models/sequential/sequential_base_model.py     153    134    12%
reco_utils/recommender/deeprec/models/sequential/sli_rec.py                    49     41    16%
reco_utils/recommender/deeprec/models/xDeepFM.py                              208     79    62%
reco_utils/recommender/fastai/__init__.py                                       0      0   100%
reco_utils/recommender/fastai/fastai_utils.py                                  31      5    84%
reco_utils/recommender/geoimc/__init__.py                                       0      0   100%
reco_utils/recommender/geoimc/geoimc_algorithm.py                              74     10    86%
reco_utils/recommender/geoimc/geoimc_data.py                                   97     56    42%
reco_utils/recommender/geoimc/geoimc_predict.py                                43      3    93%
reco_utils/recommender/geoimc/geoimc_utils.py                                  15      0   100%
reco_utils/recommender/lightfm/__init__.py                                      0      0   100%
reco_utils/recommender/lightfm/lightfm_utils.py                                74     27    64%
reco_utils/recommender/lightgbm/__init__.py                                     0      0   100%
reco_utils/recommender/lightgbm/lightgbm_utils.py                             133      0   100%
reco_utils/recommender/ncf/__init__.py                                          0      0   100%
reco_utils/recommender/ncf/dataset.py                                         113      9    92%
reco_utils/recommender/ncf/ncf_singlenode.py                                  131      2    98%
reco_utils/recommender/newsrec/__init__.py                                      0      0   100%
reco_utils/recommender/newsrec/io/__init__.py                                   0      0   100%
reco_utils/recommender/newsrec/io/mind_all_iterator.py                        262     83    68%
reco_utils/recommender/newsrec/io/mind_iterator.py                            185     60    68%
reco_utils/recommender/newsrec/models/__init__.py                               0      0   100%
reco_utils/recommender/newsrec/models/base_model.py                           163    114    30%
reco_utils/recommender/newsrec/models/layers.py                               118     55    53%
reco_utils/recommender/newsrec/models/lstur.py                                 72      9    88%
reco_utils/recommender/newsrec/models/naml.py                                 120      9    92%
reco_utils/recommender/newsrec/models/npa.py                                   70      3    96%
reco_utils/recommender/newsrec/models/nrms.py                                  61      5    92%
reco_utils/recommender/newsrec/newsrec_utils.py                                81     18    78%
reco_utils/recommender/rbm/__init__.py                                          0      0   100%
reco_utils/recommender/rbm/rbm.py                                             222     59    73%
reco_utils/recommender/rlrmc/RLRMCalgorithm.py                                127     42    67%
reco_utils/recommender/rlrmc/RLRMCdataset.py                                   57      7    88%
reco_utils/recommender/rlrmc/__init__.py                                        0      0   100%
reco_utils/recommender/rlrmc/conjugate_gradient_ms.py                         118     36    69%
reco_utils/recommender/sar/__init__.py                                          1      0   100%
reco_utils/recommender/sar/sar_singlenode.py                                  160      5    97%
reco_utils/recommender/surprise/__init__.py                                     0      0   100%
reco_utils/recommender/surprise/surprise_utils.py                              28      6    79%
reco_utils/recommender/tfidf/__init__.py                                        0      0   100%
reco_utils/recommender/tfidf/tfidf_utils.py                                   142     34    76%
reco_utils/recommender/vowpal_wabbit/__init__.py                                0      0   100%
reco_utils/recommender/vowpal_wabbit/vw.py                                     67      3    96%
reco_utils/recommender/wide_deep/__init__.py                                    0      0   100%
reco_utils/recommender/wide_deep/wide_deep_utils.py                            31      2    94%
reco_utils/tuning/__init__.py                                                   0      0   100%
reco_utils/tuning/nni/nni_utils.py                                             72     11    85%
reco_utils/tuning/parameter_sweep.py                                           16      0   100%
-----------------------------------------------------------------------------------------------
TOTAL                                                                        7347   2844    61%

Expected behavior with the suggested feature

We should target for coverage>80%

Other Comments

[miguelgfierro]

Spark utilities

pytest tests/unit/test_spark_splitter.py --cov=reco_utils/dataset/
pytest tests/unit/test_spark_evaluation.py --cov=reco_utils/evaluation/

Name                                        Stmts    Miss  Cover
----------------------------------------------------------------
reco_utils/dataset/spark_splitters.py           72      6    92%
reco_utils/dataset/split_utils.py               48     20    58%
reco_utils/evaluation/spark_evaluation.py      113     24    79%
----------------------------------------------------------------
TOTAL                          233     50    79%

[Sarah-Yifei-Wang]

I can help with this issue! Please let me know if help is still needed

[miguelgfierro]

Absolutely Sarah, everything that you can do to improve the tests is welcommed