search

JacobCallahan / rizza

An increasingly intelligent method to test RH Satellite.
Other
10 stars 6 forks source link

flaky test test_positive_nested_reach_goal and test_positive_organism #25

Open tstrych opened 2 years ago

tstrych commented 2 years ago

Run 50 times test_rizza 5 times failed - test_positive_nested_reach_goal 2 times failed - test_positive_organism

______________________________ test_positive_nested_reach_goal ______________________________

    def test_positive_nested_reach_goal():
        """Create a population and breed them until one matches """
        test_pop = genetics.Population(
            gene_base=NESTED_GENOME, population_count=10)
        for i in range(10):
            test_pop.population[i].points = grade_nested_list(
                test_pop.population[i].genes
            )
        test_pop.sort_population()
        initial_points = test_pop.population[0].points
        generations = 1
        while test_pop.population[0].points > 0 and generations < 100:
            test_pop.breed_population()
            assert len(test_pop.population) == test_pop.population_count
            generations += 1
            for i in range(10):
                test_pop.population[i].points = grade_nested_list(
                    test_pop.population[i].genes
                )
        test_pop.sort_population()
>       assert test_pop.population[0].points < initial_points
E       assert 12749855292 < 9660187854
E        +  where 12749855292 = Organism(genes=[[11, 11, 11, 76, 99], [4, 92, 66, 66, 92], [16, 16, 25, 3, 3]], points=12749855292).points

tests/helpers/test_genetics.py:92: AssertionError
__________________________________ test_positive_organism ___________________________________

    def test_positive_organism():
        test_org = genetics.Organism(genes=BASE_GENOME)
        assert test_org.genes == BASE_GENOME
        test_org.generate_genes()
        # This assertion has a 1:3628800 chance of failing
        assert test_org.genes != BASE_GENOME
        previous = test_org.genes[:]
        # We will make the mutation chance 100% to force a mutation
        test_org.mutate(gene_base=BASE_GENOME, mutation_chance=1)
>       assert test_org.genes != previous
E       assert [1, 7, 6, 14, 2, 22, ...] != [1, 7, 6, 14, 2, 22, ...]
E        +  where [1, 7, 6, 14, 2, 22, ...] = Organism(genes=[1, 7, 6, 14, 2, 22, 13, 12, 16, 17, 0, 24, 19, 10, 21, 18, 3, 5, 4, 9, 20, 23, 15, 8, 11], points=0).genes

tests/helpers/test_genetics.py:104: AssertionError
JacobCallahan commented 2 years ago

Yep, those are almost unavoidably flaky. The resolution for this issue would either be: