Improve pytest speed

[Zethson]

Description of feature

Currently, starting pytest takes ages because it doesn't collect the tests that fast or seems to download something. I don't really know what's going on.

I think this was introduced when we merged the CellLineMetaData PR.

• [ ] Figure out why test collection takes so long. Are we checking too many folders? Is it downloading things? Why are these not cached? What's happening?
• [ ] Move more fixtures/anndata object creations into conftest fixtures
• [ ] Cache cachedir on Github Actions tests
• [x] Benchmark which tests take too long. Which functions take too long? Document them.
• [ ] Discuss with @Zethson how we'll speed everything up

[Zethson]

@Lilly-May @wxicu this is for later. Please ignore it for now - you have other things to do that are currently more important.

[Zethson]

716.87s call tests/tools/test_augur.py::test_differential_prioritization 265.23s call tests/tools/_distances/test_distance_tests.py::test_distancetest[wasserstein] 91.72s call tests/tools/test_mixscape.py::test_perturbation_signature 88.49s call tests/metadata/test_cell_line.py::test_bulk_rna_expression_annotation 48.50s call tests/tools/test_mixscape.py::test_lda 46.60s call tests/tools/test_augur.py::test_random_forest_classifier 44.65s setup tests/tools/_distances/test_distances.py::test_distance_axioms 42.71s setup tests/tools/_distances/test_distances.py::test_distance_layers 42.25s setup tests/tools/_distances/test_distances.py::test_triangle_inequality 42.06s setup tests/tools/_distances/test_distances.py::test_distance_pairwise 41.77s setup tests/tools/_distances/test_distances.py::test_distance_output_type 39.17s setup tests/tools/_distances/test_distances.py::test_distance_counts 30.01s call tests/tools/_coda/test_sccoda.py::test_make_arviz 26.42s call tests/tools/test_dialogue.py::test_dialogue_pipeline 25.98s call tests/tools/_coda/test_sccoda.py::test_credible_effects 20.37s call tests/tools/_coda/test_tasccoda.py::test_run_nuts 16.22s call tests/tools/_perturbation_space/test_simple_perturbation_space.py::test_label_transfer 14.47s call tests/tools/_coda/test_sccoda.py::test_run_nuts