StanfordBDHG / XCTRuntimeAssertions

XCTest extensions to test runtime assertions and preconditions
https://swiftpackageindex.com/StanfordBDHG/XCTRuntimeAssertions/documentation/xctruntimeassertions
MIT License
1 stars 0 forks source link

XCTRuntimePrecondition results in a memory leak #13

Open Supereg opened 4 months ago

Supereg commented 4 months ago

Description

When using XCTRuntimePrecondition the precondition methods are overridden and redirected to call neverReturn() which loops forever (see here). This will result in the DispatchWorkItem created in the XCTRuntimePrecondition method to run forever. Fundamentally, this will result in all resources captured by the closures passed to precondition methods to leak over the runtime of the program (refer to here).

Reproduction

Here is a short code example on how to reproduce the issue and to illustrate how this issue is unexpected.

class Test1 {}
func method1(instance: Test1) {
    precondition(instance === Test1(), "The instances was captured in this string as well \(instance)")
}

func foo() {
    let test = Test1()
    print(CFGetRetainCount(test)) // prints 2

    try XCTRuntimePrecondition {
        method1(instance: test)
    }

    print(CFGetRetainCount(test)) // prints now 4 instead of 2
    // test will not be deallocated when returning from method foo
}

Expected behavior

Ideally it should not leak memory. Refer to additional context for a discussion.

Additional context

As the source code comment explains in neverReturn(), this seems to be the only option to produce a Never return type. The issue is more about criticizing that this side effect is not documented. Such a memory leak is unexpected especially as we are overloading standard library functions which can go unnoticed. At the same time the issue should serve as a resource to point at in documentation to explain the issue in great detail and illustrate the possible side effects in testing setups.

Code of Conduct

PSchmiedmayer commented 4 months ago

Thanks for documenting this @Supereg. Just to clarify this here: The memory leak only happens within a testing environment when using XCTRuntimePrecondition so this doesn't affect any production use cases and as test cases are very ephemeral it might not have a longer-lasting impact, even on the system under test?

Agree that some documentation around this might be very helpful!

Supereg commented 4 months ago

Thanks for documenting this @Supereg. Just to clarify this here: The memory leak only happens within a testing environment when using XCTRuntimePrecondition so this doesn't affect any production use cases and as test cases are very ephemeral it might not have a longer-lasting impact, even on the system under test?

Agree that some documentation around this might be very helpful!

Yes. Important thing to clarify. This issue is only present in a testing environment and when suing e.g. XCTRuntimePrecondition. It doesn't affect production use cases. I was actively testing deallocation of Modules in https://github.com/StanfordSpezi/Spezi/pull/105 and this is where I uncovered this issue and lost an hour debugging this. So generally, I would recommend to use XCTRuntimePrecondition test in complete isolation.