Add run metric suite and run evaluation support

[mdekstrand]

This moves forward #489 to provide item-list-based bulk evaluation of entire runs.

Right now it is probably a lot slower than the old code (no bulk optimizations), but it should be correct, and better-documented and more extensible.

[codecov[bot]]

Codecov Report

Attention: Patch coverage is 88.10289% with 37 lines in your changes missing coverage. Please review.

Project coverage is 92.55%. Comparing base (bf8606e) to head (8b88566). Report is 20 commits behind head on main.

Files with missing lines	Patch %	Lines
lenskit/lenskit/data/bulk.py	83.78%	6 Missing :warning:
lenskit/lenskit/metrics/bulk.py	92.40%	6 Missing :warning:
lenskit/lenskit/metrics/ranking/_hit.py	40.00%	6 Missing :warning:
lenskit/lenskit/metrics/ranking/_recip.py	40.00%	6 Missing :warning:
lenskit/lenskit/metrics/ranking/_pr.py	60.00%	4 Missing :warning:
lenskit/lenskit/metrics/ranking/_rbp.py	40.00%	3 Missing :warning:
lenskit/lenskit/data/items.py	84.61%	2 Missing :warning:
lenskit/lenskit/metrics/_base.py	90.90%	2 Missing :warning:
lenskit/lenskit/metrics/basic.py	92.30%	1 Missing :warning:
lenskit/lenskit/metrics/ranking/_dcg.py	80.00%	1 Missing :warning:

Additional details and impacted files

```diff @@ Coverage Diff @@ ## main #501 +/- ## ========================================== - Coverage 92.75% 92.55% -0.21% ========================================== Files 85 90 +5 Lines 5632 5897 +265 ========================================== + Hits 5224 5458 +234 - Misses 408 439 +31 ```

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

[github-actions[bot]]

The LensKit 🤖 has run the tests on your PR.

🚨 Tests covered 88.04% of diff.

🚨 Coverage decreased -0.19% from 92.52% to 92.32%.

origin/main...HEAD, staged and unstaged changes

- lenskit/lenskit/data/bulk.py (83.3%): Missing lines 42,46-48,96,127 - lenskit/lenskit/data/items.py (84.6%): Missing lines 245,247 - lenskit/lenskit/data/schemas.py (100%) - lenskit/lenskit/data/types.py (100%) - lenskit/lenskit/metrics/__init__.py (100%) - lenskit/lenskit/metrics/_base.py (95.0%): Missing lines 54 - lenskit/lenskit/metrics/basic.py (90.0%): Missing lines 29 - lenskit/lenskit/metrics/bulk.py (92.1%): Missing lines 91,101,107,110,171,173 - lenskit/lenskit/metrics/predict.py (100%) - lenskit/lenskit/metrics/ranking/__init__.py (100%) - lenskit/lenskit/metrics/ranking/_base.py (100%) - lenskit/lenskit/metrics/ranking/_dcg.py (80.0%): Missing lines 61 - lenskit/lenskit/metrics/ranking/_hit.py (40.0%): Missing lines 19-20,22,26-27,29 - lenskit/lenskit/metrics/ranking/_pr.py (60.0%): Missing lines 22,47-48,50 - lenskit/lenskit/metrics/ranking/_rbp.py (40.0%): Missing lines 50-51,53 - lenskit/lenskit/metrics/ranking/_recip.py (40.0%): Missing lines 23-24,26,30-31,33 - lenskit/lenskit/splitting/__init__.py (100%) - lenskit/lenskit/splitting/records.py (100%) - lenskit/lenskit/splitting/split.py (100%) ## Summary - **Total**: 301 lines - **Missing**: 36 lines - **Coverage**: 88%

Source Coverage Report

| Name | Stmts | Miss | Cover | |------------------------------------------------ | -------: | -------: | ------: | | lenskit/lenskit/util/envcheck.py | 57 | 44 | 23% | | lenskit/lenskit/data/fetch.py | 38 | 28 | 26% | | lenskit/lenskit/metrics/ranking/\_hit.py | 19 | 6 | 68% | | lenskit/lenskit/metrics/ranking/\_recip.py | 24 | 7 | 71% | | lenskit/lenskit/data/types.py | 48 | 13 | 73% | | lenskit/lenskit/util/\_\_init\_\_.py | 72 | 19 | 74% | | lenskit/lenskit/topn.py | 109 | 25 | 77% | | lenskit/lenskit/data/movielens.py | 97 | 18 | 81% | | lenskit/lenskit/util/test.py | 104 | 19 | 82% | | lenskit/lenskit/pipeline/state.py | 44 | 8 | 82% | | lenskit/lenskit/parallel/config.py | 65 | 11 | 83% | | lenskit/lenskit/data/bulk.py | 36 | 6 | 83% | | lenskit/lenskit/parallel/pool.py | 54 | 9 | 83% | | lenskit/lenskit/algorithms/svd.py | 75 | 12 | 84% | | lenskit/lenskit/basic/composite.py | 20 | 3 | 85% | | lenskit/lenskit/algorithms/ranking.py | 75 | 11 | 85% | | lenskit/lenskit/metrics/ranking/\_rbp.py | 29 | 4 | 86% | | lenskit/lenskit/metrics/ranking/\_pr.py | 31 | 4 | 87% | | lenskit/lenskit/metrics/ranking/\_base.py | 16 | 2 | 88% | | lenskit/lenskit/algorithms/\_\_init\_\_.py | 67 | 8 | 88% | | lenskit/lenskit/util/random.py | 34 | 4 | 88% | | lenskit-implicit/lenskit/implicit.py | 94 | 10 | 89% | | lenskit/lenskit/splitting/split.py | 19 | 2 | 89% | | lenskit/lenskit/metrics/basic.py | 10 | 1 | 90% | | lenskit/lenskit/data/items.py | 224 | 20 | 91% | | lenskit/lenskit/batch/\_recommend.py | 46 | 4 | 91% | | lenskit/lenskit/data/vocab.py | 84 | 7 | 92% | | lenskit/lenskit/metrics/bulk.py | 76 | 6 | 92% | | lenskit/lenskit/splitting/holdout.py | 56 | 4 | 93% | | lenskit/lenskit/parallel/worker.py | 43 | 3 | 93% | | lenskit/lenskit/batch/\_predict.py | 30 | 2 | 93% | | lenskit/lenskit/math/sparse.py | 60 | 4 | 93% | | lenskit/lenskit/parallel/invoker.py | 31 | 2 | 94% | | lenskit/lenskit/algorithms/knn/user.py | 177 | 11 | 94% | | lenskit/lenskit/basic/popularity.py | 34 | 2 | 94% | | lenskit/lenskit/pipeline/components.py | 51 | 3 | 94% | | lenskit/lenskit/pipeline/types.py | 86 | 5 | 94% | | lenskit/lenskit/algorithms/knn/item.py | 302 | 17 | 94% | | lenskit/lenskit/data/dataset.py | 131 | 7 | 95% | | lenskit/lenskit/data/mtarray.py | 57 | 3 | 95% | | lenskit/lenskit/metrics/predict.py | 76 | 4 | 95% | | lenskit/lenskit/data/matrix.py | 233 | 12 | 95% | | lenskit/lenskit/metrics/\_base.py | 20 | 1 | 95% | | lenskit/lenskit/parallel/chunking.py | 20 | 1 | 95% | | lenskit/lenskit/basic/bias.py | 93 | 4 | 96% | | lenskit-funksvd/lenskit/funksvd.py | 187 | 8 | 96% | | lenskit/lenskit/metrics/ranking/\_dcg.py | 51 | 2 | 96% | | lenskit/lenskit/basic/topn.py | 26 | 1 | 96% | | lenskit/lenskit/pipeline/\_\_init\_\_.py | 291 | 11 | 96% | | lenskit/lenskit/data/lazy.py | 34 | 1 | 97% | | lenskit/lenskit/algorithms/basic.py | 161 | 4 | 98% | | lenskit/lenskit/algorithms/als/explicit.py | 121 | 3 | 98% | | lenskit/lenskit/algorithms/bias.py | 150 | 3 | 98% | | lenskit/lenskit/parallel/serialize.py | 51 | 1 | 98% | | lenskit/lenskit/algorithms/als/common.py | 128 | 2 | 98% | | lenskit/lenskit/crossfold.py | 136 | 2 | 99% | | lenskit/lenskit/pipeline/config.py | 83 | 1 | 99% | | lenskit/lenskit/pipeline/runner.py | 98 | 1 | 99% | | lenskit/lenskit/algorithms/als/implicit.py | 112 | 1 | 99% | | lenskit/lenskit/metrics/topn.py | 212 | 1 | 99% | | lenskit-hpf/lenskit/hpf.py | 24 | 0 | 100% | | lenskit/lenskit/algorithms/als/\_\_init\_\_.py | 3 | 0 | 100% | | lenskit/lenskit/algorithms/knn/\_\_init\_\_.py | 3 | 0 | 100% | | lenskit/lenskit/algorithms/mf\_common.py | 61 | 0 | 100% | | lenskit/lenskit/basic/\_\_init\_\_.py | 6 | 0 | 100% | | lenskit/lenskit/basic/candidates.py | 25 | 0 | 100% | | lenskit/lenskit/basic/history.py | 60 | 0 | 100% | | lenskit/lenskit/basic/random.py | 25 | 0 | 100% | | lenskit/lenskit/batch/\_\_init\_\_.py | 2 | 0 | 100% | | lenskit/lenskit/data/\_\_init\_\_.py | 10 | 0 | 100% | | lenskit/lenskit/data/checks.py | 37 | 0 | 100% | | lenskit/lenskit/data/convert.py | 50 | 0 | 100% | | lenskit/lenskit/data/query.py | 27 | 0 | 100% | | lenskit/lenskit/data/schemas.py | 40 | 0 | 100% | | lenskit/lenskit/data/tables.py | 25 | 0 | 100% | | lenskit/lenskit/diagnostics.py | 4 | 0 | 100% | | lenskit/lenskit/math/\_\_init\_\_.py | 0 | 0 | 100% | | lenskit/lenskit/math/solve.py | 6 | 0 | 100% | | lenskit/lenskit/metrics/\_\_init\_\_.py | 16 | 0 | 100% | | lenskit/lenskit/metrics/ranking/\_\_init\_\_.py | 7 | 0 | 100% | | lenskit/lenskit/parallel/\_\_init\_\_.py | 4 | 0 | 100% | | lenskit/lenskit/parallel/sequential.py | 22 | 0 | 100% | | lenskit/lenskit/pipeline/common.py | 18 | 0 | 100% | | lenskit/lenskit/pipeline/nodes.py | 47 | 0 | 100% | | lenskit/lenskit/splitting/\_\_init\_\_.py | 4 | 0 | 100% | | lenskit/lenskit/splitting/records.py | 58 | 0 | 100% | | lenskit/lenskit/splitting/users.py | 61 | 0 | 100% | | lenskit/lenskit/types.py | 5 | 0 | 100% | | lenskit/lenskit/util/logging.py | 19 | 0 | 100% | | lenskit/lenskit/util/timing.py | 28 | 0 | 100% | | **TOTAL** | **5705** | **438** | **92%** |

[github-actions[bot]]

The LensKit 🤖 has run the tests on your PR.

🚨 Tests covered 87.71% of diff.

🚨 Coverage decreased -0.21% from 92.52% to 92.30%.

origin/main...HEAD, staged and unstaged changes

- lenskit/lenskit/data/bulk.py (83.3%): Missing lines 42,46-48,96,127 - lenskit/lenskit/data/items.py (84.6%): Missing lines 245,247 - lenskit/lenskit/data/schemas.py (100%) - lenskit/lenskit/data/types.py (100%) - lenskit/lenskit/metrics/__init__.py (100%) - lenskit/lenskit/metrics/_base.py (90.0%): Missing lines 54,57 - lenskit/lenskit/metrics/basic.py (90.0%): Missing lines 29 - lenskit/lenskit/metrics/bulk.py (92.1%): Missing lines 91,101,107,110,171,173 - lenskit/lenskit/metrics/predict.py (100%) - lenskit/lenskit/metrics/ranking/__init__.py (100%) - lenskit/lenskit/metrics/ranking/_base.py (100%) - lenskit/lenskit/metrics/ranking/_dcg.py (80.0%): Missing lines 61 - lenskit/lenskit/metrics/ranking/_hit.py (40.0%): Missing lines 19-20,22,26-27,29 - lenskit/lenskit/metrics/ranking/_pr.py (60.0%): Missing lines 22,47-48,50 - lenskit/lenskit/metrics/ranking/_rbp.py (40.0%): Missing lines 50-51,53 - lenskit/lenskit/metrics/ranking/_recip.py (40.0%): Missing lines 23-24,26,30-31,33 - lenskit/lenskit/splitting/__init__.py (100%) - lenskit/lenskit/splitting/records.py (100%) - lenskit/lenskit/splitting/split.py (100%) ## Summary - **Total**: 301 lines - **Missing**: 37 lines - **Coverage**: 87%

Source Coverage Report

| Name | Stmts | Miss | Cover | |------------------------------------------------ | -------: | -------: | ------: | | lenskit/lenskit/util/envcheck.py | 57 | 44 | 23% | | lenskit/lenskit/data/fetch.py | 38 | 28 | 26% | | lenskit/lenskit/metrics/ranking/\_hit.py | 19 | 6 | 68% | | lenskit/lenskit/metrics/ranking/\_recip.py | 24 | 7 | 71% | | lenskit/lenskit/data/types.py | 48 | 13 | 73% | | lenskit/lenskit/util/\_\_init\_\_.py | 72 | 19 | 74% | | lenskit/lenskit/topn.py | 109 | 25 | 77% | | lenskit/lenskit/data/movielens.py | 97 | 18 | 81% | | lenskit/lenskit/util/test.py | 104 | 19 | 82% | | lenskit/lenskit/pipeline/state.py | 44 | 8 | 82% | | lenskit/lenskit/parallel/config.py | 65 | 11 | 83% | | lenskit/lenskit/data/bulk.py | 36 | 6 | 83% | | lenskit/lenskit/parallel/pool.py | 54 | 9 | 83% | | lenskit/lenskit/algorithms/svd.py | 75 | 12 | 84% | | lenskit/lenskit/basic/composite.py | 20 | 3 | 85% | | lenskit/lenskit/algorithms/ranking.py | 75 | 11 | 85% | | lenskit/lenskit/metrics/ranking/\_rbp.py | 29 | 4 | 86% | | lenskit/lenskit/metrics/ranking/\_pr.py | 31 | 4 | 87% | | lenskit/lenskit/metrics/ranking/\_base.py | 16 | 2 | 88% | | lenskit/lenskit/algorithms/\_\_init\_\_.py | 67 | 8 | 88% | | lenskit/lenskit/util/random.py | 34 | 4 | 88% | | lenskit-implicit/lenskit/implicit.py | 94 | 10 | 89% | | lenskit/lenskit/splitting/split.py | 19 | 2 | 89% | | lenskit/lenskit/metrics/\_base.py | 20 | 2 | 90% | | lenskit/lenskit/metrics/basic.py | 10 | 1 | 90% | | lenskit/lenskit/data/items.py | 224 | 20 | 91% | | lenskit/lenskit/batch/\_recommend.py | 46 | 4 | 91% | | lenskit/lenskit/data/vocab.py | 84 | 7 | 92% | | lenskit/lenskit/metrics/bulk.py | 76 | 6 | 92% | | lenskit/lenskit/splitting/holdout.py | 56 | 4 | 93% | | lenskit/lenskit/parallel/worker.py | 43 | 3 | 93% | | lenskit/lenskit/batch/\_predict.py | 30 | 2 | 93% | | lenskit/lenskit/math/sparse.py | 60 | 4 | 93% | | lenskit/lenskit/parallel/invoker.py | 31 | 2 | 94% | | lenskit/lenskit/algorithms/knn/user.py | 177 | 11 | 94% | | lenskit/lenskit/basic/popularity.py | 34 | 2 | 94% | | lenskit/lenskit/pipeline/components.py | 51 | 3 | 94% | | lenskit/lenskit/pipeline/types.py | 86 | 5 | 94% | | lenskit/lenskit/algorithms/knn/item.py | 302 | 17 | 94% | | lenskit/lenskit/data/dataset.py | 131 | 7 | 95% | | lenskit/lenskit/data/mtarray.py | 57 | 3 | 95% | | lenskit/lenskit/metrics/predict.py | 76 | 4 | 95% | | lenskit/lenskit/data/matrix.py | 233 | 12 | 95% | | lenskit/lenskit/parallel/chunking.py | 20 | 1 | 95% | | lenskit/lenskit/basic/bias.py | 93 | 4 | 96% | | lenskit-funksvd/lenskit/funksvd.py | 187 | 8 | 96% | | lenskit/lenskit/metrics/ranking/\_dcg.py | 51 | 2 | 96% | | lenskit/lenskit/basic/topn.py | 26 | 1 | 96% | | lenskit/lenskit/pipeline/\_\_init\_\_.py | 291 | 11 | 96% | | lenskit/lenskit/data/lazy.py | 34 | 1 | 97% | | lenskit/lenskit/algorithms/basic.py | 161 | 4 | 98% | | lenskit/lenskit/algorithms/als/explicit.py | 121 | 3 | 98% | | lenskit/lenskit/algorithms/bias.py | 150 | 3 | 98% | | lenskit/lenskit/parallel/serialize.py | 51 | 1 | 98% | | lenskit/lenskit/algorithms/als/common.py | 128 | 2 | 98% | | lenskit/lenskit/crossfold.py | 136 | 2 | 99% | | lenskit/lenskit/pipeline/config.py | 83 | 1 | 99% | | lenskit/lenskit/pipeline/runner.py | 98 | 1 | 99% | | lenskit/lenskit/algorithms/als/implicit.py | 112 | 1 | 99% | | lenskit/lenskit/metrics/topn.py | 212 | 1 | 99% | | lenskit-hpf/lenskit/hpf.py | 24 | 0 | 100% | | lenskit/lenskit/algorithms/als/\_\_init\_\_.py | 3 | 0 | 100% | | lenskit/lenskit/algorithms/knn/\_\_init\_\_.py | 3 | 0 | 100% | | lenskit/lenskit/algorithms/mf\_common.py | 61 | 0 | 100% | | lenskit/lenskit/basic/\_\_init\_\_.py | 6 | 0 | 100% | | lenskit/lenskit/basic/candidates.py | 25 | 0 | 100% | | lenskit/lenskit/basic/history.py | 60 | 0 | 100% | | lenskit/lenskit/basic/random.py | 25 | 0 | 100% | | lenskit/lenskit/batch/\_\_init\_\_.py | 2 | 0 | 100% | | lenskit/lenskit/data/\_\_init\_\_.py | 10 | 0 | 100% | | lenskit/lenskit/data/checks.py | 37 | 0 | 100% | | lenskit/lenskit/data/convert.py | 50 | 0 | 100% | | lenskit/lenskit/data/query.py | 27 | 0 | 100% | | lenskit/lenskit/data/schemas.py | 40 | 0 | 100% | | lenskit/lenskit/data/tables.py | 25 | 0 | 100% | | lenskit/lenskit/diagnostics.py | 4 | 0 | 100% | | lenskit/lenskit/math/\_\_init\_\_.py | 0 | 0 | 100% | | lenskit/lenskit/math/solve.py | 6 | 0 | 100% | | lenskit/lenskit/metrics/\_\_init\_\_.py | 16 | 0 | 100% | | lenskit/lenskit/metrics/ranking/\_\_init\_\_.py | 7 | 0 | 100% | | lenskit/lenskit/parallel/\_\_init\_\_.py | 4 | 0 | 100% | | lenskit/lenskit/parallel/sequential.py | 22 | 0 | 100% | | lenskit/lenskit/pipeline/common.py | 18 | 0 | 100% | | lenskit/lenskit/pipeline/nodes.py | 47 | 0 | 100% | | lenskit/lenskit/splitting/\_\_init\_\_.py | 4 | 0 | 100% | | lenskit/lenskit/splitting/records.py | 58 | 0 | 100% | | lenskit/lenskit/splitting/users.py | 61 | 0 | 100% | | lenskit/lenskit/types.py | 5 | 0 | 100% | | lenskit/lenskit/util/logging.py | 19 | 0 | 100% | | lenskit/lenskit/util/timing.py | 28 | 0 | 100% | | **TOTAL** | **5705** | **439** | **92%** |

[github-actions[bot]]

The LensKit 🤖 has run the tests on your PR.

🚨 Tests covered 87.71% of diff.

🚨 Coverage decreased -0.21% from 92.52% to 92.30%.

origin/main...HEAD, staged and unstaged changes

- lenskit/lenskit/data/bulk.py (83.3%): Missing lines 42,46-48,96,127 - lenskit/lenskit/data/items.py (84.6%): Missing lines 245,247 - lenskit/lenskit/data/schemas.py (100%) - lenskit/lenskit/data/types.py (100%) - lenskit/lenskit/metrics/__init__.py (100%) - lenskit/lenskit/metrics/_base.py (90.0%): Missing lines 54,57 - lenskit/lenskit/metrics/basic.py (90.0%): Missing lines 29 - lenskit/lenskit/metrics/bulk.py (92.1%): Missing lines 91,101,107,110,171,173 - lenskit/lenskit/metrics/predict.py (100%) - lenskit/lenskit/metrics/ranking/__init__.py (100%) - lenskit/lenskit/metrics/ranking/_base.py (100%) - lenskit/lenskit/metrics/ranking/_dcg.py (80.0%): Missing lines 61 - lenskit/lenskit/metrics/ranking/_hit.py (40.0%): Missing lines 19-20,22,26-27,29 - lenskit/lenskit/metrics/ranking/_pr.py (60.0%): Missing lines 22,47-48,50 - lenskit/lenskit/metrics/ranking/_rbp.py (40.0%): Missing lines 50-51,53 - lenskit/lenskit/metrics/ranking/_recip.py (40.0%): Missing lines 23-24,26,30-31,33 - lenskit/lenskit/splitting/__init__.py (100%) - lenskit/lenskit/splitting/records.py (100%) - lenskit/lenskit/splitting/split.py (100%) ## Summary - **Total**: 301 lines - **Missing**: 37 lines - **Coverage**: 87%

Source Coverage Report

| Name | Stmts | Miss | Cover | |------------------------------------------------ | -------: | -------: | ------: | | lenskit/lenskit/util/envcheck.py | 57 | 44 | 23% | | lenskit/lenskit/data/fetch.py | 38 | 28 | 26% | | lenskit/lenskit/metrics/ranking/\_hit.py | 19 | 6 | 68% | | lenskit/lenskit/metrics/ranking/\_recip.py | 24 | 7 | 71% | | lenskit/lenskit/data/types.py | 48 | 13 | 73% | | lenskit/lenskit/util/\_\_init\_\_.py | 72 | 19 | 74% | | lenskit/lenskit/topn.py | 109 | 25 | 77% | | lenskit/lenskit/data/movielens.py | 97 | 18 | 81% | | lenskit/lenskit/util/test.py | 104 | 19 | 82% | | lenskit/lenskit/pipeline/state.py | 44 | 8 | 82% | | lenskit/lenskit/parallel/config.py | 65 | 11 | 83% | | lenskit/lenskit/data/bulk.py | 36 | 6 | 83% | | lenskit/lenskit/parallel/pool.py | 54 | 9 | 83% | | lenskit/lenskit/algorithms/svd.py | 75 | 12 | 84% | | lenskit/lenskit/basic/composite.py | 20 | 3 | 85% | | lenskit/lenskit/algorithms/ranking.py | 75 | 11 | 85% | | lenskit/lenskit/metrics/ranking/\_rbp.py | 29 | 4 | 86% | | lenskit/lenskit/metrics/ranking/\_pr.py | 31 | 4 | 87% | | lenskit/lenskit/metrics/ranking/\_base.py | 16 | 2 | 88% | | lenskit/lenskit/algorithms/\_\_init\_\_.py | 67 | 8 | 88% | | lenskit/lenskit/util/random.py | 34 | 4 | 88% | | lenskit-implicit/lenskit/implicit.py | 94 | 10 | 89% | | lenskit/lenskit/splitting/split.py | 19 | 2 | 89% | | lenskit/lenskit/metrics/\_base.py | 20 | 2 | 90% | | lenskit/lenskit/metrics/basic.py | 10 | 1 | 90% | | lenskit/lenskit/data/items.py | 224 | 20 | 91% | | lenskit/lenskit/batch/\_recommend.py | 46 | 4 | 91% | | lenskit/lenskit/data/vocab.py | 84 | 7 | 92% | | lenskit/lenskit/metrics/bulk.py | 76 | 6 | 92% | | lenskit/lenskit/splitting/holdout.py | 56 | 4 | 93% | | lenskit/lenskit/parallel/worker.py | 43 | 3 | 93% | | lenskit/lenskit/batch/\_predict.py | 30 | 2 | 93% | | lenskit/lenskit/math/sparse.py | 60 | 4 | 93% | | lenskit/lenskit/parallel/invoker.py | 31 | 2 | 94% | | lenskit/lenskit/algorithms/knn/user.py | 177 | 11 | 94% | | lenskit/lenskit/basic/popularity.py | 34 | 2 | 94% | | lenskit/lenskit/pipeline/components.py | 51 | 3 | 94% | | lenskit/lenskit/pipeline/types.py | 86 | 5 | 94% | | lenskit/lenskit/algorithms/knn/item.py | 302 | 17 | 94% | | lenskit/lenskit/data/dataset.py | 131 | 7 | 95% | | lenskit/lenskit/data/mtarray.py | 57 | 3 | 95% | | lenskit/lenskit/metrics/predict.py | 76 | 4 | 95% | | lenskit/lenskit/data/matrix.py | 233 | 12 | 95% | | lenskit/lenskit/parallel/chunking.py | 20 | 1 | 95% | | lenskit/lenskit/basic/bias.py | 93 | 4 | 96% | | lenskit-funksvd/lenskit/funksvd.py | 187 | 8 | 96% | | lenskit/lenskit/metrics/ranking/\_dcg.py | 51 | 2 | 96% | | lenskit/lenskit/basic/topn.py | 26 | 1 | 96% | | lenskit/lenskit/pipeline/\_\_init\_\_.py | 291 | 11 | 96% | | lenskit/lenskit/data/lazy.py | 34 | 1 | 97% | | lenskit/lenskit/algorithms/basic.py | 161 | 4 | 98% | | lenskit/lenskit/algorithms/als/explicit.py | 121 | 3 | 98% | | lenskit/lenskit/algorithms/bias.py | 150 | 3 | 98% | | lenskit/lenskit/parallel/serialize.py | 51 | 1 | 98% | | lenskit/lenskit/algorithms/als/common.py | 128 | 2 | 98% | | lenskit/lenskit/crossfold.py | 136 | 2 | 99% | | lenskit/lenskit/pipeline/config.py | 83 | 1 | 99% | | lenskit/lenskit/pipeline/runner.py | 98 | 1 | 99% | | lenskit/lenskit/algorithms/als/implicit.py | 112 | 1 | 99% | | lenskit/lenskit/metrics/topn.py | 212 | 1 | 99% | | lenskit-hpf/lenskit/hpf.py | 24 | 0 | 100% | | lenskit/lenskit/algorithms/als/\_\_init\_\_.py | 3 | 0 | 100% | | lenskit/lenskit/algorithms/knn/\_\_init\_\_.py | 3 | 0 | 100% | | lenskit/lenskit/algorithms/mf\_common.py | 61 | 0 | 100% | | lenskit/lenskit/basic/\_\_init\_\_.py | 6 | 0 | 100% | | lenskit/lenskit/basic/candidates.py | 25 | 0 | 100% | | lenskit/lenskit/basic/history.py | 60 | 0 | 100% | | lenskit/lenskit/basic/random.py | 25 | 0 | 100% | | lenskit/lenskit/batch/\_\_init\_\_.py | 2 | 0 | 100% | | lenskit/lenskit/data/\_\_init\_\_.py | 10 | 0 | 100% | | lenskit/lenskit/data/checks.py | 37 | 0 | 100% | | lenskit/lenskit/data/convert.py | 50 | 0 | 100% | | lenskit/lenskit/data/query.py | 27 | 0 | 100% | | lenskit/lenskit/data/schemas.py | 40 | 0 | 100% | | lenskit/lenskit/data/tables.py | 25 | 0 | 100% | | lenskit/lenskit/diagnostics.py | 4 | 0 | 100% | | lenskit/lenskit/math/\_\_init\_\_.py | 0 | 0 | 100% | | lenskit/lenskit/math/solve.py | 6 | 0 | 100% | | lenskit/lenskit/metrics/\_\_init\_\_.py | 16 | 0 | 100% | | lenskit/lenskit/metrics/ranking/\_\_init\_\_.py | 7 | 0 | 100% | | lenskit/lenskit/parallel/\_\_init\_\_.py | 4 | 0 | 100% | | lenskit/lenskit/parallel/sequential.py | 22 | 0 | 100% | | lenskit/lenskit/pipeline/common.py | 18 | 0 | 100% | | lenskit/lenskit/pipeline/nodes.py | 47 | 0 | 100% | | lenskit/lenskit/splitting/\_\_init\_\_.py | 4 | 0 | 100% | | lenskit/lenskit/splitting/records.py | 58 | 0 | 100% | | lenskit/lenskit/splitting/users.py | 61 | 0 | 100% | | lenskit/lenskit/types.py | 5 | 0 | 100% | | lenskit/lenskit/util/logging.py | 19 | 0 | 100% | | lenskit/lenskit/util/timing.py | 28 | 0 | 100% | | **TOTAL** | **5705** | **439** | **92%** |