kif-framework / KIF

Keep It Functional - An iOS Functional Testing Framework
Other
6.21k stars 914 forks source link

Crash on acknowledgeSystemAlert #956

Closed AncAinu closed 6 years ago

AncAinu commented 7 years ago

image

(lldb) p [alert isVisible]
error: Execution was interrupted, reason: EXC_BAD_INSTRUCTION (code=EXC_I386_INVOP, subcode=0x0).
The process has been returned to the state before expression evaluation.

(lldb) p alert
(UIAAlert *) $0 = 0x7b9cc340

(lldb) p [alert isValid]
(char) $1 = '\x01'

I receive a EXC_BAD_INSTRUCTION on the isVisible but the UIAAlert is populated, and isValid does work.

The crash is nearly systematic, although sometimes it doesn't crash and does close the alert.

passchaos commented 7 years ago

I think you are acknowledge alerts triggered by app not system alert. You can use [alert parentElement] to judge app alert or system alert. Use tap... to acknowledge app alert.

AncAinu commented 7 years ago

@passchaos my alert is indeed a UIAlertController, although I can get to actually find that view, accessibility labels seems ignored, and I didn't find any doc about it (or 2 years old and not working) If you have 2 mins could you tell me the approach to have ?

passchaos commented 7 years ago

@AncAinu Using alert text is OK. Try [tester tapViewWithAccessibilityLabel:].

billburgess commented 7 years ago

I'm having the same issue when using this. I have a number of tests that surround getting permission to use Push Notifications. It seems to fail sporadically. I'm still trying to figure out what is causing the issue and the order, but this method is exactly what I need. I don't know if/when the system alert will be presented and need it tapped and closed if it does show. Let me know if I can provide any other information on this to help figure out what is causing the crash.

Here is the output from the debugger when checking the application and alert.

(lldb) po application

(lldb) po application.alert error: Execution was interrupted, reason: EXC_BAD_INSTRUCTION (code=EXC_I386_INVOP, subcode=0x0). The process has been returned to the state before expression evaluation.
AncAinu commented 7 years ago

@billburgess I actually found one issue which was related in our case. It's that the try/catch done with KIF (i.e. find if the popup exists) are leading to memory leak and our popup wasn't released, which would make the whole thing crash since we rely on it to be released.

billburgess commented 7 years ago

Is this something I can patch in and use until a fix is out... or not fixable?

AncAinu commented 7 years ago

@billburgess sadly it's the usage of try/catch itself which creates memory leaks, try to avoid using that

justinseanmartin commented 7 years ago

@billburgess - Memory leaks from try/catch are generally caused by this: http://clang.llvm.org/docs/AutomaticReferenceCounting.html#exceptions

KIF itself shouldn't be throwing with any references to your application objects. XCTest will catch thrown exceptions though (whether raised by the app or by a KIF test failure). I think this is unrelated to what you're seeing however.

What version of Xcode are you using? Do you have a sample project that reproduces the failure?

If you're not explicitly testing the system alert and not receiving pushes received from the server, I'd personally recommend stubbing out -[UIApplication registerForRemoteNotifications] (e.g. swizzle it to be a no-op).

I personally don't like using the Apple UIAutomation library being part of KIF, though I know it serves a purpose. I know it works (sort of), but having this black box library dependency that changes based on Xcode version seems prone to hitting these types of issues.

billburgess commented 7 years ago

I worked around it by implementing applesimutils and just suppressing the system alert from showing at all.

phatmann commented 7 years ago

@billburgess Can you provide more details?

billburgess commented 7 years ago

@phatmann It is an open source library that lets you set the permissions on the simulator before your app is even installed or run. I only use it as part of my build script on CircleCI. Essentially, start the simulator, set the permissions, run my tests. This avoids needing to add the code to "maybe" dismiss the system alert if it comes up. This way all of my tests can run without fear of the alert. I can mock and work around the permissions as I wish after that. https://github.com/wix/AppleSimulatorUtils

justinseanmartin commented 6 years ago

Closing stale issue. I'm not sure if this crash is still a problem or not, but I'd personally like to purge all of the code that uses UIAutomation.framework from KIF if it isn't working. Using wix/AppleSimulatorUtils seems like a reasonable workaround to initialize the simulator state with permissions granted if needed.