search

mozilla / jetstream

This machine kills superstition. 💨
https://experimenter.info/jetstream/jetstream
Mozilla Public License 2.0
23 stars 18 forks source link

Slow tests #791

Open data-sync-user opened 3 years ago

data-sync-user commented 3 years ago

These tests are slow and suspicious for network access.

tox -r -e py38 -- --durations=0

6.43s call     .tox/py38/lib/python3.8/site-packages/jetstream/tests/test_analysis.py::test_regression_20200316
2.28s call     .tox/py38/lib/python3.8/site-packages/jetstream/tests/test_external_config.py::TestExternalConfig::test_from_github_repo
1.87s call     .tox/py38/lib/python3.8/site-packages/jetstream/tests/test_metadata.py::test_metadata_from_config

┆Issue is synchronized with this Jira Task

data-sync-user commented 3 years ago

➤ Tim D. Smith commented:

I removed a Github access from test_regression_20200316; it's still slow because spinning up Dask seems to have some waits somewhere, which is fine. I can't see it touching the network with tcpdump.

The other tests are hitting the default OutcomesResolver.

data-sync-user commented 3 years ago

➤ Anna Scholtz commented:

One other thing we can try to speed up pytest is try out pytest-xdist ( https://pypi.org/project/pytest-xdist/ ) to run tests in parallel

data-sync-user commented 3 years ago

➤ Tim D. Smith commented:

I tried it locally and it actually hurt a little bit because of the extra setup. I'm not as worried about speed as I am about isolation; "slow" is just a proxy here for "hitting the network."