[tests] test_slow_example fails sporadically

[stanislavlevin]

interegular 0.3.2.

test_slow_example test randomly fails on different architectures. For example,

=================================== FAILURES =================================== 
_______________________ test_slow_example[SLOW_EXAMPLE] ________________________ 

comp = <interegular.comparator.Comparator object at 0x7fffbc230550>              
expected = (('EXIT_TAG', 'COMMENT'),)                                            

    @pytest.mark.parametrize("comp, expected", [                                 
        pytest.param(('EXIT_TAG', 'COMMENT'), (('EXIT_TAG', 'COMMENT'),), id="SLOW_EXAMPLE")
    ], indirect=['comp'])                                                        
    def test_slow_example(comp, expected):                                       
        for collision in expected:                                               
            start = perf_counter()                                               
            assert not comp.isdisjoint(*collision)                               
            try:                                                                 
                example = comp.get_example_overlap(*collision, 0.5)              
                assert comp.get_fsm(collision[0]).accepts(example)               
                assert comp.get_fsm(collision[1]).accepts(example)               
            except ValueError:                                                   
                pass                                                             
            end = perf_counter()                                                 
>           assert end - start < 1                                               
E           assert (1617056.717598456 - 1617055.271879278) < 1                   

tests/test_comparator.py:62: AssertionError

Is this expected flaky test?

max_iterations = int((max_time - 0.09)/(1.4e-6 * len(alphabet)))

[MegaIng]

This is more or less expected, yes :-/ The problem is that different machines vary widely in the types of performance it can achieved, as well as python version mattering quite a bit. You can try to play around with the parameters to calculate max_iterations, but I doubt that general good parameters will be findable.

An alternative would be to regularly recheck the time inside of the inner strings method, but I am not sure if I like that better.