Setup test coverage (text & html reports available in CI)

[cmyui]

Describe your changes

This PR adds test coverage to our test suite. Coverage allows us to see what code flows we are testing, and helps identify untested code.

Related Issues / Projects

Checklist

• [x] I've manually tested my code

[cmyui]

Basic coverage report is working:

======================== 85 passed, 11 xfailed in 5.67s ========================
Name                                    Stmts   Miss  Cover
-----------------------------------------------------------
app/__init__.py                            10      0   100%
app/_typing.py                             17      4    76%
app/api/__init__.py                         9      0   100%
app/api/domains/__init__.py                 3      0   100%
app/api/domains/cho.py                   1054    683    35%
app/api/domains/map.py                      9      1    89%
app/api/domains/osu.py                    639    323    49%
app/api/init_api.py                       100     22    78%
app/api/middlewares.py                     22      0   100%
app/api/v1/__init__.py                      4      0   100%
app/api/v1/api.py                         282    215    24%
app/api/v2/__init__.py                     10      0   100%
app/api/v2/clans.py                        23      9    61%
app/api/v2/common/json.py                  19      9    53%
app/api/v2/common/responses.py             24      6    75%
app/api/v2/maps.py                         23      9    61%
app/api/v2/models/__init__.py              11      1    91%
app/api/v2/models/clans.py                  9      0   100%
app/api/v2/models/maps.py                  27      0   100%
app/api/v2/models/players.py               43      0   100%
app/api/v2/models/scores.py                23      0   100%
app/api/v2/players.py                      47     23    51%
app/api/v2/scores.py                       23      9    61%
app/bg_loops.py                            42     14    67%
app/commands.py                          1317   1020    23%
app/constants/__init__.py                   5      0   100%
app/constants/clientflags.py               35      0   100%
app/constants/gamemodes.py                 45      6    87%
app/constants/mods.py                     121     55    55%
app/constants/privileges.py                38      0   100%
app/constants/regexes.py                    9      0   100%
app/discord.py                            114     77    32%
app/encryption.py                          16      0   100%
app/objects/__init__.py                     9      0   100%
app/objects/achievement.py                 15      7    53%
app/objects/beatmap.py                    [353](https://github.com/osuAkatsuki/bancho.py/actions/runs/7926964078/job/21642556207#step:5:354)    140    60%
app/objects/channel.py                     60     25    58%
app/objects/clan.py                        39     25    36%
app/objects/collections.py                244    121    50%
app/objects/match.py                      314    216    31%
app/objects/models.py                       5      0   100%
app/objects/player.py                     522    268    49%
app/objects/score.py                      204     70    66%
app/packets.py                            621    141    77%
app/repositories/__init__.py                0      0   100%
app/repositories/achievements.py           98     59    40%
app/repositories/channels.py               81     49    40%
app/repositories/clans.py                  75     44    41%
app/repositories/client_hashes.py          39      2    95%
app/repositories/comments.py               36     12    67%
app/repositories/favourites.py             31     16    48%
app/repositories/ingame_logins.py          50     17    66%
app/repositories/logs.py                   24      8    67%
app/repositories/mail.py                   40     17    58%
app/repositories/map_requests.py           36     19    47%
app/repositories/maps.py                  150     83    45%
app/repositories/players.py               120     55    54%
app/repositories/ratings.py                34     20    41%
app/repositories/scores.py                 97     32    67%
app/repositories/stats.py                 115     26    77%
app/repositories/user_achievements.py      29      3    90%
app/settings_utils.py                      23     10    57%
app/state/__init__.py                      16      3    81%
app/state/cache.py                         10      2    80%
app/state/services.py                     228    119    48%
app/state/sessions.py                      33      5    85%
app/usecases/__init__.py                    0      0   100%
app/usecases/achievements.py                9      2    78%
app/usecases/performance.py                57      2    96%
app/usecases/user_achievements.py          11      0   100%
app/utils.py                              185     79    57%
-----------------------------------------------------------
TOTAL                                    8186   4183    49%

I will now see if I can get an html report setup as an artifact export that's easily accessible for engineers in CI & locally

[cmyui]

A bit awkward to find & use, but html reports are now available in CI

Perhaps we can do better? Would be nice to have these served on a public url. cc @NiceAesth @tsunyoku in case you have any ideas

[tsunyoku]

i know at work that our coverage workflow automatically errors if the coverage has decreased since a previous commit.. is that not possible here? would be better than hard-coding numbers

[cmyui]

i know at work that our coverage workflow automatically errors if the coverage has decreased since a previous commit.. is that not possible here? would be better than hard-coding numbers

Hmm.. Not that I can find built-in: https://coverage.readthedocs.io/en/7.4.1/config.html#report-fail-under

It would need to involve feeding the previous metrics (metrics on the base branch) to the coverage tool somehow.. I suspect we could do it in theory via --fail-under=${prev_pct}, but it's not something I've explored before.

would be better than hard-coding numbers

Yeah especially for when we're at these new-suite stages. Once we're at 100%, it matters less.. But there's a long path ahead 🤣

[cmyui]

Going to merge for now