swiftlang / swift-corelibs-xctest

The XCTest Project, A Swift core library for providing unit test support
swift.org
Apache License 2.0
1.15k stars 267 forks source link

await fulfillment(of: [exp] does take more time than needed. #464

Closed agiokas closed 1 year ago

agiokas commented 1 year ago

A quite strange behaviour when using the await fulfillment(of: [exp])

    func testWaitForValue_Exp() async {
        let exp = expectation(description: "My Expectation")
        Task {
            exp.fulfill()
        }

        await fulfillment(of: [exp], timeout: 1)
    }

when using a timeout bigger than 0.1 sec most tests take around 0.1 sec to complete.

Test Case '-[XYZ testWaitForValue_Exp]' started (Iteration 91 of 100).
Test Case '-[XYZ testWaitForValue_Exp]' passed (0.001 seconds).
Test Case '-[XYZ testWaitForValue_Exp]' started (Iteration 92 of 100).
Test Case '-[XYZ testWaitForValue_Exp]' passed (0.001 seconds).
Test Case '-[XYZ testWaitForValue_Exp]' started (Iteration 93 of 100).
Test Case '-[XYZ testWaitForValue_Exp]' passed (0.104 seconds).
Test Case '-[XYZ testWaitForValue_Exp]' started (Iteration 94 of 100).
Test Case '-[XYZ testWaitForValue_Exp]' passed (0.003 seconds).
Test Case '-[XYZ testWaitForValue_Exp]' started (Iteration 95 of 100).
Test Case '-[XYZ testWaitForValue_Exp]' passed (0.001 seconds).
Test Case '-[XYZ testWaitForValue_Exp]' started (Iteration 96 of 100).
Test Case '-[XYZ testWaitForValue_Exp]' passed (0.001 seconds).
Test Case '-[XYZ testWaitForValue_Exp]' started (Iteration 97 of 100).
Test Case '-[XYZ testWaitForValue_Exp]' passed (0.103 seconds).
Test Case '-[XYZ testWaitForValue_Exp]' started (Iteration 98 of 100).
Test Case '-[XYZ testWaitForValue_Exp]' passed (0.103 seconds).
Test Case '-[XYZ testWaitForValue_Exp]' started (Iteration 99 of 100).
Test Case '-[XYZ testWaitForValue_Exp]' passed (0.103 seconds).
Test Case '-[XYZ testWaitForValue_Exp]' started (Iteration 100 of 100).
Test Case '-[XYZ testWaitForValue_Exp]' passed (0.103 seconds).

when using a timeout under 0.1 sec most tests need the timeout duration to complete

    func testWaitForValue_Exp() async {
        let exp = expectation(description: "My Expectation")
        Task {
            exp.fulfill()
        }

        await fulfillment(of: [exp], timeout: 0.02)
    }
Test Case '-[XYZ testWaitForValue_Exp]' started (Iteration 94 of 100).
Test Case '-[XYZ testWaitForValue_Exp]' passed (0.001 seconds).
Test Case '-[XYZ testWaitForValue_Exp]' started (Iteration 95 of 100).
Test Case '-[XYZ testWaitForValue_Exp]' passed (0.024 seconds).
Test Case '-[XYZ testWaitForValue_Exp]' started (Iteration 96 of 100).
Test Case '-[XYZ testWaitForValue_Exp]' passed (0.025 seconds).
Test Case '-[XYZ testWaitForValue_Exp]' started (Iteration 97 of 100).
Test Case '-[XYZ testWaitForValue_Exp]' passed (0.001 seconds).
Test Case '-[XYZ testWaitForValue_Exp]' started (Iteration 98 of 100).
Test Case '-[XYZ testWaitForValue_Exp]' passed (0.024 seconds).
Test Case '-[XYZ testWaitForValue_Exp]' started (Iteration 99 of 100).
Test Case '-[XYZ testWaitForValue_Exp]' passed (0.025 seconds).
Test Case '-[XYZ testWaitForValue_Exp]' started (Iteration 100 of 100).
Test Case '-[XYZ testWaitForValue_Exp]' passed (0.025 seconds).

Reproducible on iOS and on the Mac using Xcode 15 on Xcode 14.3.1 it works as expected.

agiokas commented 1 year ago

It is an Xcode issue, problem is not reproducible using swift test just in xcode.