search

lbl-anp / becquerel

Becquerel is a Python package for analyzing nuclear spectroscopic measurements.
Other
43 stars 16 forks source link

Run tests in parallel with `pytest-xdist` #336

Closed jvavrek closed 2 years ago

jvavrek commented 2 years ago

@jvavrek can we change the tests to use xdist and run in parallel?

Originally posted by @jccurtis in https://github.com/lbl-anp/becquerel/issues/333#issuecomment-1087928469

I was unable to get pytest-xdist to work properly—it seems to cause race conditions when used in conjunction with pytest-cov. Perhaps we can revisit this in the future. See the original post for further discussion on the original attempt.

RobJY commented 2 years ago

I found this post on SO that suggests adding --dist=loadfile to the call to pytest to get all the tests in a given file to run on the same worker. I tried this on a Docker container running the Python 3.9 image and went from getting multiple failed tests to none. I'm only running on a small number of cores so maybe it still fails with more cores. Seems like this should help though if not solve the issue.

Happy to test on other versions of Python or share my Dockerfile if it would help.

jccurtis commented 2 years ago

@RobJY that is a great find. Thank you for that! @jvavrek want me to make a separate PR with just feature?

jvavrek commented 2 years ago

@RobJY that is a great find. Thank you for that! @jvavrek want me to make a separate PR with just feature?

@jccurtis if you have time for it. If not, I can try it this afternoon.

jccurtis commented 2 years ago

@jvavrek see #337