search

appium / appium

Cross-platform automation framework for all kinds of apps, built on top of the W3C WebDriver protocol
http://appium.io/
Apache License 2.0
18.23k stars 6.01k forks source link

[Feat]: ios 18 x Xcode 16 stuff #20239

Open KazuCocoa opened 2 weeks ago

KazuCocoa commented 2 weeks ago

Not yet iOS 18/next Xcode is available, so this is a placeholder for it right now.

Just a note which could be used for automation

Hmm, potentially, the Swift testing will be the primary framework? Our WDA inherits XCTestCase so it will affect the WDA in the future...?

For Swift Testing: In case the swift testing will be a primary and they drop XCTest syntax, our WDA may need to rewrite in Swift and Swift Testing basis ? (https://github.com/appium/WebDriverAgent/blob/4684a2a56980c9d6038f7218b511a6f0f0207ad6/WebDriverAgentLib/Utilities/FBFailureProofTestCase.h#L17-L18)

https://developer.apple.com/documentation/testing/migratingfromxctest

Safari/WebKit

Tests

KazuCocoa commented 2 weeks ago

Xcode 16 beta was able to build WDA. A new session request succeeded with an ios 18 simulator on my local. I haven't checked other details well yet.

Safari also worked, but the simulator (especially when I launched Safari) was extremely slow on my env compared with iOS 17 simulators. Perhaps I needed to stop all other program? but that could cause unexpected errors by the host machine's slowness. So, I think our testing should wait a bit to be tools being more stable to avoid false negatives.

Anyway, so far, my minimal quick check did not find critical issues.

I haven't checked with real devices yet.

mykola-mokhnach commented 2 weeks ago

It looks like this API only targets WatchOS

Hmm, potentially, the Swift testing will be the primary framework? Our WDA inherits XCTestCase so it will affect the WDA in the future...?

For Swift Testing: In case the swift testing will be a primary and they drop XCTest syntax, our WDA may need to rewrite in Swift and Swift Testing basis ? (https://github.com/appium/WebDriverAgent/blob/4684a2a56980c9d6038f7218b511a6f0f0207ad6/WebDriverAgentLib/Utilities/FBFailureProofTestCase.h#L17-L18)

https://developer.apple.com/documentation/testing/migratingfromxctest

I assume this is only made for Switft. So we are safe unless Apple stops supporting ObjectiveC

Dan-Maor commented 2 weeks ago

@KazuCocoa looks like the Safari in the simulator is a known issue: https://developer.apple.com/documentation/ios-ipados-release-notes/ios-ipados-18-release-notes#Simulator There’s a suggestion in the release notes to use Erase All Contents and Settings to make it work properly.

KazuCocoa commented 2 weeks ago

I tested a few last night with an iPadOS 18. The UI was pretty slow in some cases, so I brought the OS version back to 17 for my regular use.

example output by remote debugger:

[HTTP]
[RemoteDebugger] Page changed: {
[RemoteDebugger]   "1": {
[RemoteDebugger]     "WIROverrideNameKey": "",
[RemoteDebugger]     "WIRConnectionIdentifierKey": "23f9e11a-2711-46fe-b3fc-8982913b9b8f",
[RemoteDebugger]     "WIRTypeKey": "WIRTypeWebPage",
[RemoteDebugger]     "WIRPageIdentifierKey": 1,
[RemoteDebugger]     "WIRTitleKey": "",
[RemoteDebugger]     "WIRURLKey": "http://127.0.0.1:8100/health",
[RemoteDebugger]     "WIRHostApplicationIdentifierKey": "PID:841"
[RemoteDebugger]   }
[RemoteDebugger] }
[XCUITestDriver@f771 (5e0501fc)] Remote debugger notified us of a new page listing: {"appIdKey":"841","pageArray":[{"id":1,"title":"","url":"http://127.0.0.1:8100/health","isKey":true}]}
[XCUITestDriver@f771 (5e0501fc)] Checking if page needs to load
[XCUITestDriver@f771 (5e0501fc)] New page listing is same as old, doing nothing
[RemoteDebugger] Notified that new application 'PID:857' has connected
[RemoteDebugger] Received page change notice for app 'PID:841' but the listing has not changed. Ignoring.
[RemoteDebugger] Page changed: {
[RemoteDebugger]   "1": {
[RemoteDebugger]     "WIROverrideNameKey": "",
[RemoteDebugger]     "WIRConnectionIdentifierKey": "23f9e11a-2711-46fe-b3fc-8982913b9b8f",
[RemoteDebugger]     "WIRTypeKey": "WIRTypeWebPage",
[RemoteDebugger]     "WIRPageIdentifierKey": 1,
[RemoteDebugger]     "WIRTitleKey": "",
[RemoteDebugger]     "WIRURLKey": "http://127.0.0.1:8100/health",
[RemoteDebugger]     "WIRHostApplicationIdentifierKey": "PID:841"
[RemoteDebugger]   }
[RemoteDebugger] }
[XCUITestDriver@f771 (5e0501fc)] Remote debugger notified us of a new page listing: {"appIdKey":"841","pageArray":[{"id":1,"title":"","url":"http://127.0.0.1:8100/health","isKey":true}]}
[XCUITestDriver@f771 (5e0501fc)] Checking if page needs to load
[XCUITestDriver@f771 (5e0501fc)] New page listing is same as old, doing nothing
[HTTP] --> POST /session/5e0501fc-0807-4de0-99ba-4ead40387fca/url
[HTTP] {"url":"https://youtube.com"}
[XCUITestDriver@f771 (5e0501fc)] Calling AppiumDriver.setUrl() with args: ["https://youtube.com","5e0501fc-0807-4de0-99ba-4ead40387fca"]
[XCUITestDriver@f771 (5e0501fc)] Executing command 'setUrl'
[XCUITestDriver@f771 (5e0501fc)] Attempting to set url 'https://youtube.com'
[RemoteDebugger] Navigating to new URL: 'https://youtube.com'
[RemoteDebugger] Sending '_rpc_forwardSocketData:' message to app 'PID:841', page '1', target 'page-81' (id: 40): 'Page.navigate'
[RemoteDebugger] Received data response from send (id: 40): '{}'
[RemoteDebugger] Sending to Web Inspector took 25ms
[RemoteDebugger] Timed out after 6000ms of waiting for the https://youtube.com page readiness. Continuing anyway
[RemoteDebugger] Sending '_rpc_forwardSocketData:' message to app 'PID:841', page '1', target 'page-81' (id: 42): 'Console.enable'
[RemoteDebugger] Sending to Web Inspector took 2ms
[XCUITestDriver@f771 (5e0501fc)] Responding to client with driver.setUrl() result: null
[HTTP] <-- POST /session/5e0501fc-0807-4de0-99ba-4ead40387fca/url 200 6011 ms - 14
[HTTP]
[HTTP] --> GET /session/5e0501fc-0807-4de0-99ba-4ead40387fca/contexts
[HTTP] {}
[XCUITestDriver@f771 (5e0501fc)] Calling AppiumDriver.getContexts() with args: ["5e0501fc-0807-4de0-99ba-4ead40387fca"]
[XCUITestDriver@f771 (5e0501fc)] Executing command 'getContexts'
[XCUITestDriver@f771 (5e0501fc)] Getting list of available contexts
[XCUITestDriver@f771 (5e0501fc)] Retrieving contexts and views
[XCUITestDriver@f771 (5e0501fc)] Selecting by url: false
[RemoteDebugger] Selecting application
[RemoteDebugger] Current applications available:
[RemoteDebugger]     Application: "PID:306"
[RemoteDebugger]         id: "PID:306"
[RemoteDebugger]         isProxy: false
[RemoteDebugger]         name: "HeuristicInterpreter"
[RemoteDebugger]         bundleId: "process-HeuristicInterpreter"
[RemoteDebugger]         hostId: undefined
[RemoteDebugger]         isActive: false
[RemoteDebugger]         isAutomationEnabled: "Unknown"
[RemoteDebugger]     Application: "PID:199"
[RemoteDebugger]         id: "PID:199"
[RemoteDebugger]         isProxy: false
[RemoteDebugger]         name: "amsengagementd"
[RemoteDebugger]         bundleId: "com.apple.amsengagementd"
[RemoteDebugger]         hostId: undefined
[RemoteDebugger]         isActive: false
[RemoteDebugger]         isAutomationEnabled: "Unknown"
[RemoteDebugger]     Application: "PID:841"
[RemoteDebugger]         id: "PID:841"
[RemoteDebugger]         isProxy: false
[RemoteDebugger]         name: "Safari"
[RemoteDebugger]         bundleId: "com.apple.mobilesafari"
[RemoteDebugger]         hostId: undefined
[RemoteDebugger]         isActive: true
[RemoteDebugger]         isAutomationEnabled: "Unknown"
[RemoteDebugger]         pageArray:
[RemoteDebugger]           - id: 1
[RemoteDebugger]             title: ""
[RemoteDebugger]             url: "http://127.0.0.1:8100/health"
[RemoteDebugger]             isKey: true
[RemoteDebugger]     Application: "PID:523"
[RemoteDebugger]         id: "PID:523"
[RemoteDebugger]         isProxy: false
[RemoteDebugger]         name: "Settings"
[RemoteDebugger]         bundleId: "com.apple.Preferences"
[RemoteDebugger]         hostId: undefined
[RemoteDebugger]         isActive: false
[RemoteDebugger]         isAutomationEnabled: "Unknown"
[RemoteDebugger]     Application: "PID:857"
[RemoteDebugger]         id: "PID:857"
[RemoteDebugger]         isProxy: false
[RemoteDebugger]         name: "reversetemplated"
[RemoteDebugger]         bundleId: "process-reversetemplated"
[RemoteDebugger]         hostId: undefined
[RemoteDebugger]         isActive: false
[RemoteDebugger]         isAutomationEnabled: "Unknown"
[RemoteDebugger]         pageArray: "Waiting for data"
[RemoteDebugger] Checking for bundle identifiers: com.apple.WebKit.WebContent, process-com.apple.WebKit.WebContent, process-SafariViewService, com.apple.SafariViewService, *, com.apple.mobilesafari
[RemoteDebugger] Found app id key 'PID:841' for bundle 'com.apple.mobilesafari'
[RemoteDebugger] Trying out the possible app ids: PID:841 (try #1 of 20)
[RemoteDebugger] Attempting app 'PID:841'
[RemoteDebugger] Sending '_rpc_forwardGetListing:' message to app 'PID:841' (id: 44): 'connectToApp'
[RemoteDebugger] Received response from send (id: 44): '["PID:841",{"1":{"WIROverrideNameKey":"","WIRConnectionIdentifierKey":"23f9e11a-2711-46fe-b3fc-8982913b9b8f","WIRTypeKey":"WIRTypeWebPage","WIRPageIdentifierKey":1,"WIRTitleKey":"","WIRURLKey":"htt...'
[RemoteDebugger] Received page change notice for app 'PID:841' but the listing has not changed. Ignoring.
[RemoteDebugger] Page changed: {
[RemoteDebugger]   "1": {
[RemoteDebugger]     "WIROverrideNameKey": "",
[RemoteDebugger]     "WIRConnectionIdentifierKey": "23f9e11a-2711-46fe-b3fc-8982913b9b8f",
[RemoteDebugger]     "WIRTypeKey": "WIRTypeWebPage",
[RemoteDebugger]     "WIRPageIdentifierKey": 1,
[RemoteDebugger]     "WIRTitleKey": "",
[RemoteDebugger]     "WIRURLKey": "http://127.0.0.1:8100/health",
[RemoteDebugger]     "WIRHostApplicationIdentifierKey": "PID:841"
[RemoteDebugger]   }
[RemoteDebugger] }
[XCUITestDriver@f771 (5e0501fc)] Remote debugger notified us of a new page listing: {"appIdKey":"841","pageArray":[{"id":1,"title":"","url":"http://127.0.0.1:8100/health","isKey":true}]}
[XCUITestDriver@f771 (5e0501fc)] Checking if page needs to load
[XCUITestDriver@f771 (5e0501fc)] New page listing is same as old, doing nothing
[XCUITestDriver@f771 (5e0501fc)] Redirected from 'https://youtube.com' to 'http://127.0.0.1:8100/health'
[RemoteDebugger] Sending to Web Inspector took 9ms
[RemoteDebugger] Current applications available:
[RemoteDebugger]     Application: "PID:306"
[RemoteDebugger]         id: "PID:306"
[RemoteDebugger]         isProxy: false
[RemoteDebugger]         name: "HeuristicInterpreter"
[RemoteDebugger]         bundleId: "process-HeuristicInterpreter"
[RemoteDebugger]         hostId: undefined
[RemoteDebugger]         isActive: false
[RemoteDebugger]         isAutomationEnabled: "Unknown"
[RemoteDebugger]     Application: "PID:199"
[RemoteDebugger]         id: "PID:199"
[RemoteDebugger]         isProxy: false
[RemoteDebugger]         name: "amsengagementd"
[RemoteDebugger]         bundleId: "com.apple.amsengagementd"
[RemoteDebugger]         hostId: undefined
[RemoteDebugger]         isActive: false
[RemoteDebugger]         isAutomationEnabled: "Unknown"
[RemoteDebugger]     Application: "PID:841"
[RemoteDebugger]         id: "PID:841"
[RemoteDebugger]         isProxy: false
[RemoteDebugger]         name: "Safari"
[RemoteDebugger]         bundleId: "com.apple.mobilesafari"
[RemoteDebugger]         hostId: undefined
[RemoteDebugger]         isActive: true
[RemoteDebugger]         isAutomationEnabled: "Unknown"
[RemoteDebugger]         pageArray:
[RemoteDebugger]           - id: 1
[RemoteDebugger]             title: ""
[RemoteDebugger]             url: "http://127.0.0.1:8100/health"
[RemoteDebugger]             isKey: true
[RemoteDebugger]     Application: "PID:523"
[RemoteDebugger]         id: "PID:523"
[RemoteDebugger]         isProxy: false
[RemoteDebugger]         name: "Settings"
[RemoteDebugger]         bundleId: "com.apple.Preferences"
[RemoteDebugger]         hostId: undefined
[RemoteDebugger]         isActive: false
[RemoteDebugger]         isAutomationEnabled: "Unknown"
[RemoteDebugger]     Application: "PID:857"
[RemoteDebugger]         id: "PID:857"
[RemoteDebugger]         isProxy: false
[RemoteDebugger]         name: "reversetemplated"
[RemoteDebugger]         bundleId: "process-reversetemplated"
[RemoteDebugger]         hostId: undefined
[RemoteDebugger]         isActive: false
[RemoteDebugger]         isAutomationEnabled: "Unknown"
[RemoteDebugger]         pageArray: "Waiting for data"
[RemoteDebugger] Finally selecting app PID:841: [{"id":1,"title":"","url":"http://127.0.0.1:8100/health","isKey":true}]
[RemoteDebugger] Selected app after 14ms
[XCUITestDriver@f771 (5e0501fc)] Responding to client with driver.getContexts() result: ["NATIVE_APP","WEBVIEW_841.1"]
[HTTP] <-- GET /session/5e0501fc-0807-4de0-99ba-4ead40387fca/contexts 200 21 ms - 40
KazuCocoa commented 2 weeks ago

Ran ruby_lib_core's test suites for iPhone 18/tvOS 18 simulators. They looked ok.

KazuCocoa commented 2 weeks ago

I did more testing on a real device. Most of the things I checked worked. (Updated this description) One thing was a file push. Maybe... https://github.com/appium/appium-xcuitest-driver/pull/2409 which was a regression, so it is not iOS 18 related

mykola-mokhnach commented 2 weeks ago

I did more testing on a real device. Most of the things I checked worked. (Updated this description) One thing was a file push. Maybe... appium/appium-xcuitest-driver#2409 which was a regression, so it is not iOS 18 related

Thank you for checking this Kazu

candialala commented 1 week ago
image

I tested it on simulator18.0 and couldn't get the alert elements.

KazuCocoa commented 1 week ago

Please refer to https://appium.github.io/appium-xcuitest-driver/latest/guides/troubleshooting/#interact-with-dialogs-managed-by-comapplespringboard

krishtoautomate commented 1 day ago

FYI: https://x.com/srtg000bot/status/1801567034934968550/video/1

apple is exposing its HID api even to allow control on real devices like simulators