search

realm / SwiftCov

A tool to generate test code coverage information for Swift.
MIT License
561 stars 41 forks source link

Async networking expectation output error or sometimes never complete when running SwiftCov #39

Closed sync closed 9 years ago

sync commented 9 years ago

this:

func testAsyncNetworking() {
        let e = self.expectationWithDescription("complete expectation")

        let session = NSURLSession.sharedSession()
        let task = session.dataTaskWithURL(NSURL(string: "http://www.google.com")!, completionHandler: { (data, response, error) -> Void in
             e.fulfill()
        })
        task.resume()

        self.waitForExpectationsWithTimeout(2, handler: nil)
    }

error:

Test Case '-[ExampleFrameworkiOSTests.ExampleFrameworkTests testAsyncNetworking]' started.
2015-06-03 13:45:25.021 xctest[23736:1435239] ***storageTaskManagerExistsWithIdentifier:withIdentifier failed: Error Domain=NSCocoaErrorDomain Code=4099 "The operation couldn’t be completed. (Cocoa error 4099.)" (The connection to service named com.apple.nsurlstorage-cache was invalidated.) UserInfo=0x7fd9fa667b40 {NSDebugDescription=The connection to service named com.apple.nsurlstorage-cache was invalidated.}; {
    NSDebugDescription = "The connection to service named com.apple.nsurlstorage-cache was invalidated.";
}
2015-06-03 13:45:25.022 xctest[23736:1435239] ***cachedResponseForRequest:key:handler failed: Error Domain=NSCocoaErrorDomain Code=4099 "The operation couldn’t be completed. (Cocoa error 4099.)" (The connection to service named com.apple.nsurlstorage-cache was invalidated.) UserInfo=0x7fd9fa706710 {NSDebugDescription=The connection to service named com.apple.nsurlstorage-cache was invalidated.}; {
    NSDebugDescription = "The connection to service named com.apple.nsurlstorage-cache was invalidated.";
}
2015-06-03 13:45:25.064 xctest[23736:1435243] ***cachedResponseForRequest:key:handler failed: Error Domain=NSCocoaErrorDomain Code=4099 "The operation couldn’t be completed. (Cocoa error 4099.)" (The connection to service named com.apple.nsurlstorage-cache was invalidated.) UserInfo=0x7fd9fa7072a0 {NSDebugDescription=The connection to service named com.apple.nsurlstorage-cache was invalidated.}; {
    NSDebugDescription = "The connection to service named com.apple.nsurlstorage-cache was invalidated.";
}

see https://github.com/sync/SwiftCov/tree/bug/async-networking-crash

sync commented 9 years ago

how:

this is the output when it never complete:

Generate test code coverage files
Loading build settings...
Building target...
Generating code coverage...
error: libarclite_iphonesimulator.a(arclite.o) failed to load objfile for /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/arc/libarclite_iphonesimulator.a
error: libarclite_iphonesimulator.a(arclite.o) failed to load objfile for /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/arc/libarclite_iphonesimulator.a
Test Suite 'Selected tests' started at 2015-06-03 03:52:01 +0000
Test Suite 'Quick.framework' started at 2015-06-03 03:52:01 +0000
Test Suite 'Quick.framework' passed at 2015-06-03 03:52:01 +0000.
     Executed 0 tests, with 0 failures (0 unexpected) in 0.000 (0.000) seconds
Test Suite 'LoadFacebookProfileKitTests.xctest' started at 2015-06-03 03:52:01 +0000
Test Suite 'FBSDKGraphRequestTegExtensionsSpec' started at 2015-06-03 03:52:01 +0000
Test Case '-[LoadFacebookProfileKitTests.FBSDKGraphRequestTegExtensionsSpec on_failure__should_return_an_error]' started.
2015-06-03 13:52:02.009 xctest[24666:1468637] ***storageTaskManagerExistsWithIdentifier:withIdentifier failed: Error Domain=NSCocoaErrorDomain Code=4099 "The operation couldn’t be completed. (Cocoa error 4099.)" (The connection to service named com.apple.nsurlstorage-cache was invalidated.) UserInfo=0x7ffb1162cbe0 {NSDebugDescription=The connection to service named com.apple.nsurlstorage-cache was invalidated.}; {
    NSDebugDescription = "The connection to service named com.apple.nsurlstorage-cache was invalidated.";
}
2015-06-03 13:52:02.065 xctest[24666:1468637] ***cachedResponseForRequest:key:handler failed: Error Domain=NSCocoaErrorDomain Code=4099 "The operation couldn’t be completed. (Cocoa error 4099.)" (The connection to service named com.apple.nsurlstorage-cache was invalidated.) UserInfo=0x7ffb1166de20 {NSDebugDescription=The connection to service named com.apple.nsurlstorage-cache was invalidated.}; {
    NSDebugDescription = "The connection to service named com.apple.nsurlstorage-cache was invalidated.";
}
2015-06-03 13:52:02.104 xctest[24666:1468640] NSURLConnection/CFURLConnection HTTP load failed (kCFStreamErrorDomainSSL, -9807)
sonsongithub commented 9 years ago

I got as same bug as yours.

Mac-mini:sample sonson$ swiftcov generate --output ./coverage --threshold 1 xcodebuild test -project sample.xcodeproj -scheme 'sample' -configuration Release -sdk iphonesimulator
Generate test code coverage files
Loading build settings...
Building target...
Generating code coverage...
Test Suite 'Selected tests' started at 2015-06-03 10:41:06 +0000
Test Suite 'sampleTests.xctest' started at 2015-06-03 10:41:06 +0000
Test Suite 'sampleTests' started at 2015-06-03 10:41:06 +0000
Test Case '-[sampleTests.sampleTests testExample]' started.
2015-06-03 19:41:06.314 xctest[51283:9371487] ***storageTaskManagerExistsWithIdentifier:withIdentifier failed: Error Domain=NSCocoaErrorDomain Code=4099 "The operation couldn’t be completed. (Cocoa error 4099.)" (The connection to service named com.apple.nsurlstorage-cache was invalidated.) UserInfo=0x7f8ac05ee100 {NSDebugDescription=The connection to service named com.apple.nsurlstorage-cache was invalidated.}; {
    NSDebugDescription = "The connection to service named com.apple.nsurlstorage-cache was invalidated.";
}
2015-06-03 19:41:06.315 xctest[51283:9371487] ***cachedResponseForRequest:key:handler failed: Error Domain=NSCocoaErrorDomain Code=4099 "The operation couldn’t be completed. (Cocoa error 4099.)" (The connection to service named com.apple.nsurlstorage-cache was invalidated.) UserInfo=0x7f8ac0622b10 {NSDebugDescription=The connection to service named com.apple.nsurlstorage-cache was invalidated.}; {
    NSDebugDescription = "The connection to service named com.apple.nsurlstorage-cache was invalidated.";
}
2015-06-03 19:41:26.304 xctest[51283:9371325] Unable to identify simulator root path while running on the simulator
Invalid connection: com.apple.coresymbolicationd
/Users/sonson/Desktop/sample/sampleTests/sampleTests.swift:30: error: -[sampleTests.sampleTests testExample] : Asynchronous wait failed: Exceeded timeout of 20 seconds, with unfulfilled expectations: "Test : Getting OAuth2 access token".
Test Case '-[sampleTests.sampleTests testExample]' failed (20.036 seconds).
Test Suite 'sampleTests' failed at 2015-06-03 10:41:26 +0000.
     Executed 1 test, with 1 failure (0 unexpected) in 20.036 (20.036) seconds
Test Suite 'sampleTests.xctest' failed at 2015-06-03 10:41:26 +0000.
     Executed 1 test, with 1 failure (0 unexpected) in 20.036 (20.037) seconds
Test Suite 'Selected tests' failed at 2015-06-03 10:41:26 +0000.
     Executed 1 test, with 1 failure (0 unexpected) in 20.036 (20.038) seconds
Collecting coverage data...
Generating gcov files...
File: '/Users/sonson/Desktop/sample/sample/Hoge.swift'
Lines executed: 70.00% of 10
Created './coverage/Hoge.swift.gcov'
Coverage files successfully generated to './coverage'

This error is reproduced using this project

kishikawakatsumi commented 9 years ago

Thank you for reporting. I've found the way to avoid the issue.

To solve the issue, add an environment variable before launching tests as follows:

XPC_SIMULATOR_LAUNCHD_NAME="com.apple.CoreSimulator.SimDevice.04042E78-E743-4376-94E9-31842D4770B6.launchd_sim"

However, the 04042E78-E743-4376-94E9-31842D4770B6 is depends on each environment. So it can not be hard-coded. I'm looking for a way to read the value from project settings.

sync commented 9 years ago

fantastic thanks

segiddins commented 9 years ago

@kishikawakatsumi xcrun simctl list might be what you want?

kishikawakatsumi commented 9 years ago

@segiddins Yeah, hopefully it'll be work well. Thanks.

sonsongithub commented 9 years ago

Ooop, I couldn't run the test.

Is it such as the following UUID?

> xcrun simctl list
~
== Devices ==
-- iOS 8.3 --
    iPhone 4s (<MY UUID>) (Shutdown)
~

Is the following comment right?

> export XPC_SIMULATOR_LAUNCHD_NAME="com.apple.CoreSimulator.SimDevice.<MY UUID>.launchd_sim"
> swiftcov
kishikawakatsumi commented 9 years ago

Sorry my explanation was ambiguous. The env value should be pass xctest command that is called by swiftcov internally. So to fix this, needs editing the library code.

sonsongithub commented 9 years ago

I got it.

kishikawakatsumi commented 9 years ago

Thanks @segiddins simctl list works very well.

@sync @sonsongithub Thank you reporting issue and provide test cases. It was fixes https://github.com/realm/SwiftCov/issues/44 and released as 0.1.1