pytest flaky interaction causing fixture exceptions to be ignored

[timyhou]

Somewhat related to this issue which was supposedly resolved in Flaky 3.1.1

In the example below, something in flaky is causing the exception that is thrown inside the fixture not to be reported anymore.

import pytest
import flaky

@pytest.fixture
def failing_fixture():
    raise Exception("Woops")

@flaky.flaky()
@pytest.mark.usefixtures('failing_fixture')
class TestCases:
    def test_fixture_fail(self):
        pass

    def test_always_pass(self):
        assert True

    def test_always_fail(self):
        assert False

The test results show that 3 tests were gathered but none were executed (which is true) but leaves no trace to the exception in the fixture.

platform linux -- Python 3.5.1, pytest-3.0.6, py-1.4.32, pluggy-0.4.0
rootdir: /home/thou/mine/python_scribbles, inifile: 
plugins: flaky-3.3.0
collected 3 items 

my_pytest.py 
===Flaky Test Report===

===End Flaky Test Report===

 no tests ran in 0.02 seconds

If we remove the flaky decorator from the test class, the error is reported normally.

[OlehKuzovkov]

Hi guys! I found really helpful this plugging, however because of this issue I cannot use it. In case of failure in the fixture, test case marked as passed :(

@pytest.fixture
def test_fixture():
    raise Exception("Oops")

@pytest.mark.usefixtures("test_fixture")
@flaky(max_runs=2)
class TestFlaky:

    def test(self, test_fixture):
        print("test executed")
        assert False, 'Test 1 failed'

    def test_2(self):
        print("test 2 executed")
        assert False, 'Test 2 failed'

Could you please let us know when it will be fixed?