How perform appium interactions action on macOS using python

[nghianhan]

Hi, i'm new in automation on macOS How i can interactions action on macOS example: double click, right click and press key (press command + backspace). I used W3C Actions but nothing change.

[KazuCocoa]

Could you share the server log? Potentially the command itself was ok, but XCTest did not work expectedly

[nghianhan]

Thank for reply. this is log server. I used Appium 2 macOX 13.0 and python 3.9.6

log.odt

[nghianhan]

[Appium] Welcome to Appium v2.0.0-beta.47 (REV 759ab2d0bdc6b0d1f810d6e9351c834d027809f7) [Appium] Attempting to load driver mac2... [debug] [Appium] Requiring driver at /Users/admin/.appium/node_modules/appium-mac2-driver [Appium] Appium REST http interface listener started on 0.0.0.0:4723 [Appium] Available drivers: [Appium] - mac2@1.4.8 (automationName 'Mac2') [Appium] No plugins have been installed. Use the "appium plugin" command to install the one(s) you want to use. [debug] [HTTP] Request idempotency key: 818bd2c5-0e48-476a-96ce-d508b209f9c4 [HTTP] --> POST /session [HTTP] {"capabilities":{"firstMatch":[{}],"alwaysMatch":{"appium:automationName":"Mac2","platformName":"Mac","appium:bundleId":"com.apple.finder"}}} [debug] [AppiumDriver@a1a7] Calling AppiumDriver.createSession() with args: [null,null,{"firstMatch":[{}],"alwaysMatch":{"appium:automationName":"Mac2","platformName":"Mac","appium:bundleId":"com.apple.finder"}}] [debug] [AppiumDriver@a1a7] Event 'newSessionRequested' logged at 1671071604871 (11:33:24 GMT+0900 (Japan Standard Time)) [Appium] Attempting to find matching driver for automationName 'Mac2' and platformName 'Mac' [Appium] The 'mac2' driver was installed and matched caps. [Appium] Will require it at /Users/admin/.appium/node_modules/appium-mac2-driver [debug] [Appium] Requiring driver at /Users/admin/.appium/node_modules/appium-mac2-driver [AppiumDriver@a1a7] Appium v2.0.0-beta.47 creating new Mac2Driver (v1.4.8) session [AppiumDriver@a1a7] Checking BaseDriver versions for Appium and Mac2Driver [AppiumDriver@a1a7] Appium's BaseDriver version is 9.0.0 [AppiumDriver@a1a7] Mac2Driver's BaseDriver version is 8.7.3 [debug] [Mac2Driver@d227] Creating session with W3C capabilities: { [debug] [Mac2Driver@d227] "alwaysMatch": { [debug] [Mac2Driver@d227] "platformName": "Mac", [debug] [Mac2Driver@d227] "appium:automationName": "Mac2", [debug] [Mac2Driver@d227] "appium:bundleId": "com.apple.finder" [debug] [Mac2Driver@d227] }, [debug] [Mac2Driver@d227] "firstMatch": [ [debug] [Mac2Driver@d227] {} [debug] [Mac2Driver@d227] ] [debug] [Mac2Driver@d227] } [Mac2Driver@d227 (d8308acd)] Session created with session id: d8308acd-8046-4945-b3b1-f6256f15184f [debug] [WebDriverAgentMac] Using bootstrap root: /Users/admin/.appium/node_modules/appium-mac2-driver/WebDriverAgentMac [debug] [WebDriverAgentMac] Using xcodebuild binary at '/usr/bin/xcodebuild' [WebDriverAgentMac] WebDriverAgent sources are up to date (1670986522055 >= 1670986522055) [debug] [WebDriverAgentMac] Using 127.0.0.1 as server host [debug] [WebDriverAgentMac] Using port 10100 [WebDriverAgentMac] Mac2Driver host process logging is disabled. All the xcodebuild output is going to be suppressed. Set the 'showServerLogs' capability to 'true' if this is an undesired behavior [WebDriverAgentMac] Starting Mac2Driver host process: xcodebuild build-for-testing test-without-building -project /Users/admin/.appium/node_modules/appium-mac2-driver/WebDriverAgentMac/WebDriverAgentMac.xcodeproj -scheme WebDriverAgentRunner COMPILER_INDEX_STORE_ENABLE\=NO [debug] [WD Proxy] Matched '/status' to command name 'getStatus' [debug] [WD Proxy] Proxying [GET /status] to [GET http://127.0.0.1:10100/status] with no body [WD Proxy] connect ECONNREFUSED 127.0.0.1:10100 [debug] [WD Proxy] Matched '/status' to command name 'getStatus' [debug] [WD Proxy] Proxying [GET /status] to [GET http://127.0.0.1:10100/status] with no body [WD Proxy] connect ECONNREFUSED 127.0.0.1:10100 [debug] [WD Proxy] Matched '/status' to command name 'getStatus' [debug] [WD Proxy] Proxying [GET /status] to [GET http://127.0.0.1:10100/status] with no body [debug] [WD Proxy] Got response with status 200: {"value":{"message":"WebDriverAgent is ready to accept commands","state":"success","os":{"version":"Version 13.0 (Build 22A380)"},"ready":true,"build":{"time":"Dec 14 2022 12:00:40"}},"sessionId":null} [WebDriverAgentMac] The host process is ready within 2124ms [debug] [WD Proxy] Matched '/session' to command name 'createSession' [debug] [WD Proxy] Proxying [POST /session] to [POST http://127.0.0.1:10100/session] with body: {"capabilities":{"firstMatch":[{}],"alwaysMatch":{"platformName":"Mac","automationName":"Mac2","bundleId":"com.apple.finder"}}} [debug] [WD Proxy] Got response with status 200: {"value":{"sessionId":"8F30F085-9F99-45BB-B681-CB9E955E3519","capabilities":{"CFBundleIdentifier":"com.apple.finder"}},"sessionId":"8F30F085-9F99-45BB-B681-CB9E955E3519"} [WD Proxy] Determined the downstream protocol as 'W3C' [AppiumDriver@a1a7] New Mac2Driver session created successfully, session d8308acd-8046-4945-b3b1-f6256f15184f added to master session list [debug] [AppiumDriver@a1a7] Event 'newSessionStarted' logged at 1671071608574 (11:33:28 GMT+0900 (Japan Standard Time)) [debug] [Mac2Driver@d227 (d8308acd)] Cached the protocol value 'W3C' for the new session d8308acd-8046-4945-b3b1-f6256f15184f [debug] [Mac2Driver@d227 (d8308acd)] Responding to client with driver.createSession() result: {"capabilities":{"platformName":"Mac","automationName":"Mac2","bundleId":"com.apple.finder"}} [HTTP] <-- POST /session 200 3708 ms - 154 [HTTP] [HTTP] --> POST /session/d8308acd-8046-4945-b3b1-f6256f15184f/element [HTTP] {"using":"-ios class chain","value":"/XCUIElementTypeStaticText[value == \"Desktop\"]"} [debug] [Mac2Driver@d227 (d8308acd)] Calling AppiumDriver.findElement() with args: ["-ios class chain","/XCUIElementTypeStaticText[value == \"Desktop\"]","d8308acd-8046-4945-b3b1-f6256f15184f"] [debug] [Mac2Driver@d227 (d8308acd)] Valid locator strategies for this request: id, name, accessibility id, xpath, class name, -ios predicate string, predicate string, -ios class chain, class chain [debug] [WD Proxy] Matched '/element' to command name 'findElement' [debug] [WD Proxy] Proxying [POST /element] to [POST http://127.0.0.1:10100/session/8F30F085-9F99-45BB-B681-CB9E955E3519/element] with body: {"using":"class chain","value":"/XCUIElementTypeStaticText[value == \"Desktop\"]"} [debug] [WD Proxy] Got response with status 200: {"value":{"ELEMENT":"F83CD93A-863C-415C-8A62-69BBA192AC5B","element-6066-11e4-a52e-4f735466cecf":"F83CD93A-863C-415C-8A62-69BBA192AC5B"},"sessionId":"8F30F085-9F99-45BB-B681-CB9E955E3519"} [debug] [Mac2Driver@d227 (d8308acd)] Responding to client with driver.findElement() result: {"element-6066-11e4-a52e-4f735466cecf":"F83CD93A-863C-415C-8A62-69BBA192AC5B","ELEMENT":"F83CD93A-863C-415C-8A62-69BBA192AC5B"} [HTTP] <-- POST /session/d8308acd-8046-4945-b3b1-f6256f15184f/element 200 161 ms - 137 [HTTP] [HTTP] --> POST /session/d8308acd-8046-4945-b3b1-f6256f15184f/element/F83CD93A-863C-415C-8A62-69BBA192AC5B/click [HTTP] {"id":"F83CD93A-863C-415C-8A62-69BBA192AC5B"} [Mac2Driver@d227 (d8308acd)] Driver proxy active, passing request on via HTTP proxy [debug] [WD Proxy] Matched '/session/d8308acd-8046-4945-b3b1-f6256f15184f/element/F83CD93A-863C-415C-8A62-69BBA192AC5B/click' to command name 'click' [debug] [WD Proxy] Proxying [POST /session/d8308acd-8046-4945-b3b1-f6256f15184f/element/F83CD93A-863C-415C-8A62-69BBA192AC5B/click] to [POST http://127.0.0.1:10100/session/8F30F085-9F99-45BB-B681-CB9E955E3519/element/F83CD93A-863C-415C-8A62-69BBA192AC5B/click] with body: {"id":"F83CD93A-863C-415C-8A62-69BBA192AC5B"} [debug] [WD Proxy] Got response with status 200: {"value":null,"sessionId":"8F30F085-9F99-45BB-B681-CB9E955E3519"} [WD Proxy] Replacing sessionId 8F30F085-9F99-45BB-B681-CB9E955E3519 with d8308acd-8046-4945-b3b1-f6256f15184f [HTTP] <-- POST /session/d8308acd-8046-4945-b3b1-f6256f15184f/element/F83CD93A-863C-415C-8A62-69BBA192AC5B/click 200 634 ms - 65 [HTTP] [HTTP] --> POST /session/d8308acd-8046-4945-b3b1-f6256f15184f/element [HTTP] {"using":"-ios class chain","value":"/XCUIElementTypeBrowser[label == \"column view\"]/XCUIElementTypeScrollView/XCUIElementTypeScrollView/XCUIElementTypeOther/XCUIElementTypeGroup[15]"} [debug] [Mac2Driver@d227 (d8308acd)] Calling AppiumDriver.findElement() with args: ["-ios class chain","/XCUIElementTypeBrowser[label == \"column view\"]/XCUIElementTypeScrollView/XCUIElementTypeScrollView/XCUIElementTypeOther/XCUIElementTypeGroup[15]","d8308acd-8046-4945-b3b1-f6256f15184f"] [debug] [Mac2Driver@d227 (d8308acd)] Valid locator strategies for this request: id, name, accessibility id, xpath, class name, -ios predicate string, predicate string, -ios class chain, class chain [debug] [WD Proxy] Matched '/element' to command name 'findElement' [debug] [WD Proxy] Proxying [POST /element] to [POST http://127.0.0.1:10100/session/8F30F085-9F99-45BB-B681-CB9E955E3519/element] with body: {"using":"class chain","value":"/XCUIElementTypeBrowser[label == \"column view\"]/XCUIElementTypeScrollView/XCUIElementTypeScrollView/XCUIElementTypeOther/XCUIElementTypeGroup[15]"} [debug] [WD Proxy] Got response with status 200: {"value":{"ELEMENT":"D4462799-14C5-4307-8CF9-9EBB4B1E8205","element-6066-11e4-a52e-4f735466cecf":"D4462799-14C5-4307-8CF9-9EBB4B1E8205"},"sessionId":"8F30F085-9F99-45BB-B681-CB9E955E3519"} [debug] [Mac2Driver@d227 (d8308acd)] Responding to client with driver.findElement() result: {"element-6066-11e4-a52e-4f735466cecf":"D4462799-14C5-4307-8CF9-9EBB4B1E8205","ELEMENT":"D4462799-14C5-4307-8CF9-9EBB4B1E8205"} [HTTP] <-- POST /session/d8308acd-8046-4945-b3b1-f6256f15184f/element 200 125 ms - 137 [HTTP] [HTTP] --> POST /session/d8308acd-8046-4945-b3b1-f6256f15184f/element/D4462799-14C5-4307-8CF9-9EBB4B1E8205/click [HTTP] {"id":"D4462799-14C5-4307-8CF9-9EBB4B1E8205"} [Mac2Driver@d227 (d8308acd)] Driver proxy active, passing request on via HTTP proxy [debug] [WD Proxy] Matched '/session/d8308acd-8046-4945-b3b1-f6256f15184f/element/D4462799-14C5-4307-8CF9-9EBB4B1E8205/click' to command name 'click' [debug] [WD Proxy] Proxying [POST /session/d8308acd-8046-4945-b3b1-f6256f15184f/element/D4462799-14C5-4307-8CF9-9EBB4B1E8205/click] to [POST http://127.0.0.1:10100/session/8F30F085-9F99-45BB-B681-CB9E955E3519/element/D4462799-14C5-4307-8CF9-9EBB4B1E8205/click] with body: {"id":"D4462799-14C5-4307-8CF9-9EBB4B1E8205"} [debug] [WD Proxy] Got response with status 200: {"value":null,"sessionId":"8F30F085-9F99-45BB-B681-CB9E955E3519"} [WD Proxy] Replacing sessionId 8F30F085-9F99-45BB-B681-CB9E955E3519 with d8308acd-8046-4945-b3b1-f6256f15184f [HTTP] <-- POST /session/d8308acd-8046-4945-b3b1-f6256f15184f/element/D4462799-14C5-4307-8CF9-9EBB4B1E8205/click 200 560 ms - 65 [HTTP] [HTTP] --> POST /session/d8308acd-8046-4945-b3b1-f6256f15184f/actions [HTTP] {"actions":[{"type":"pointer","parameters":{"pointerType":"mouse"},"id":"mouse","actions":[{"type":"pointerMove","duration":250,"x":0,"y":0,"origin":{"element-6066-11e4-a52e-4f735466cecf":"D4462799-14C5-4307-8CF9-9EBB4B1E8205"}},{"type":"pointerMove","duration":250,"x":0,"y":0,"origin":{"element-6066-11e4-a52e-4f735466cecf":"D4462799-14C5-4307-8CF9-9EBB4B1E8205"}},{"type":"pointerDown","duration":0,"button":0},{"type":"pointerUp","duration":0,"button":0},{"type":"pointerDown","duration":0,"button":0},{"type":"pointerUp","duration":0,"button":0}]},{"type":"key","id":"key","actions":[{"type":"pause","duration":0},{"type":"pause","duration":0},{"type":"pause","duration":0},{"type":"pause","duration":0},{"type":"pause","duration":0},{"type":"pause","duration":0}]}]} [Mac2Driver@d227 (d8308acd)] Driver proxy active, passing request on via HTTP proxy [debug] [WD Proxy] Matched '/session/d8308acd-8046-4945-b3b1-f6256f15184f/actions' to command name 'performActions' [debug] [WD Proxy] Proxying [POST /session/d8308acd-8046-4945-b3b1-f6256f15184f/actions] to [POST http://127.0.0.1:10100/session/8F30F085-9F99-45BB-B681-CB9E955E3519/actions] with body: {"actions":[{"type":"pointer","parameters":{"pointerType":"mouse"},"id":"mouse","actions":[{"type":"pointerMove","duration":250,"x":0,"y":0,"origin":{"ELEMENT":"D4462799-14C5-4307-8CF9-9EBB4B1E8205","element-6066-11e4-a52e-4f735466cecf":"D4462799-14C5-4307-8CF9-9EBB4B1E8205"}},{"type":"pointerMove","duration":250,"x":0,"y":0,"origin":{"ELEMENT":"D4462799-14C5-4307-8CF9-9EBB4B1E8205","element-6066-11e4-a52e-4f735466cecf":"D4462799-14C5-4307-8CF9-9EBB4B1E8205"}},{"type":"pointerDown","duration":0,"button":0},{"type":"pointerUp","duration":0,"button":0},{"type":"pointerDown","duration":0,"button":0},{"type":"pointerUp","duration":0,"button":0}]},{"type":"key","id":"key","actions":[{"type":"pause","duration":0},{"type":"pause","duration":0},{"type":"pause","duration":0},{"type":"pause","duration":0},{"type":"pause","duration":0},{"type":"pause","duration":0}]}]} [WD Proxy] undefined [HTTP] <-- POST /session/d8308acd-8046-4945-b3b1-f6256f15184f/actions 400 240041 ms - 1056 [HTTP] [HTTP] --> DELETE /session/d8308acd-8046-4945-b3b1-f6256f15184f [HTTP] {} [debug] [Mac2Driver@d227 (d8308acd)] Calling AppiumDriver.deleteSession() with args: ["d8308acd-8046-4945-b3b1-f6256f15184f"] [debug] [AppiumDriver@a1a7] Event 'quitSessionRequested' logged at 1671071850200 (11:37:30 GMT+0900 (Japan Standard Time)) [AppiumDriver@a1a7] Removing session d8308acd-8046-4945-b3b1-f6256f15184f from our master session list [debug] [WD Proxy] Matched '/session/8F30F085-9F99-45BB-B681-CB9E955E3519' to command name 'deleteSession' [debug] [WD Proxy] Proxying [DELETE /session/8F30F085-9F99-45BB-B681-CB9E955E3519] to [DELETE http://127.0.0.1:10100/session/8F30F085-9F99-45BB-B681-CB9E955E3519] with no body [Mac2Driver@d227 (d8308acd)] Shutting down because we waited 60 seconds for a command [AppiumDriver@a1a7] Ending session, cause was 'New Command Timeout of 60 seconds expired. Try customizing the timeout using the 'newCommandTimeout' desired capability' [AppiumDriver@a1a7] Removing session 'd8308acd-8046-4945-b3b1-f6256f15184f' from our master session list [debug] [WD Proxy] Matched '/session/8F30F085-9F99-45BB-B681-CB9E955E3519' to command name 'deleteSession' [debug] [WD Proxy] Proxying [DELETE /session/8F30F085-9F99-45BB-B681-CB9E955E3519] to [DELETE http://127.0.0.1:10100/session/8F30F085-9F99-45BB-B681-CB9E955E3519] with no body [WD Proxy] undefined [WebDriverAgentMac] Mac2Driver session cannot be deleted. Original error: An unknown server-side error occurred while processing the command. Original error: Could not proxy command to the remote server. Original error: timeout of 240000ms exceeded [debug] [AppiumDriver@a1a7] Event 'quitSessionFinished' logged at 1671072090201 (11:41:30 GMT+0900 (Japan Standard Time)) [debug] [AppiumDriver@a1a7] Received response: null [debug] [AppiumDriver@a1a7] But deleting session, so not returning [debug] [AppiumDriver@a1a7] Responding to client with driver.deleteSession() result: null [HTTP] <-- DELETE /session/d8308acd-8046-4945-b3b1-f6256f15184f 200 240009 ms - 14 [HTTP] [WD Proxy] undefined [WebDriverAgentMac] Mac2Driver session cannot be deleted. Original error: An unknown server-side error occurred while processing the command. Original error: Could not proxy command to the remote server. Original error: timeout of 240000ms exceeded

[KazuCocoa]

 [HTTP] <-- POST /session/d8308acd-8046-4945-b3b1-f6256f15184f/actions 400 240041 ms - 1056

Thanks. The response was over 240 sec. Then, maybe the action did not occur in the XCTest internal. (I assume Appium waited for the response by XCTest).

[nghianhan]

Thanks. I have use driver.execute_script(macOS: 'action') and success interactions action. Temporarily, I can carry on. But i want to know, Appium ActionChains can interact with Mac2Driver normally or has any library support interact action with macOS X desktop applications?

[KazuCocoa]

macOS: commands call XCTest API such as https://github.com/appium/appium-mac2-driver#macos-scroll https://developer.apple.com/documentation/xctest/xcuielement/1500758-scrollbydeltax?language=objc, so they are via provided APIs. (not sure how stable for desktop applications though. In my few experience, XCTest framework for macOS is more unstable than iOS).

Appium/WDA achieves it via pointer event etc as https://github.com/appium/appium-mac2-driver/blob/master/WebDriverAgentMac/WebDriverAgentLib/Utilities/FBW3CActionsSynthesizer.m https://github.com/appium/appium-mac2-driver/blob/master/WebDriverAgentMac/IntegrationTests/AMW3CActionsTests.m is the test code to build commands. (Theoretically it should work like iOS.)

[nghianhan]

Thanks for reply. I have one more question. How i can implement wait conditions(ex: waitElementVisible) on macOS?

[KazuCocoa]

It is basically how a client implements. https://github.com/appium/appium-mac2-driver#element-attributes hittable is probably closer for visible in XCTest's term in this driver

[nghianhan]

Thank you very much @KazuCocoa for support. Hence closing the issue.