search

lbl-anp / becquerel

Becquerel is a Python package for analyzing nuclear spectroscopic measurements.
Other
43 stars 16 forks source link

Flaky NNDC tests failing #332

Closed jccurtis closed 2 years ago

jccurtis commented 2 years ago

We still have the flaky tests failing in the CI. See this pipeline

jvavrek commented 2 years ago

Those tests are indeed marked with flaky but it's possible they failed all three times if the NNDC server was down for a few seconds:

@pytest.mark.flaky(reruns=3)
class TestDecayRadiationQuery(NNDCQueryTests):
jccurtis commented 2 years ago

@jvavrek can we all a timeout between reruns?

jvavrek commented 2 years ago

Oh and the package is not pytest-flaky but pytest-rerunfailures, which is in requirements-dev.txt.

jccurtis commented 2 years ago

Maybe we can add the following to try three times with a 2 second delay only for 

@pytest.mark.flaky(reruns=3, reruns_delay=2, only_rerun=NNDCRequestError)

(Not sure on the only_rerun syntax - it might need to be a string or a list of exceptions.)

jvavrek commented 2 years ago

Maybe we can add the following to try three times with a 2 second delay only for

@pytest.mark.flaky(reruns=3, reruns_delay=2, only_rerun=NNDCRequestError)

(Not sure on the only_rerun syntax - it might need to be a string or a list of exceptions.)

I don't think that's supported yet: https://github.com/pytest-dev/pytest-rerunfailures/issues/164