can't find any text fields "unable to find an element using 'class name', value 'XCUIElementTypeTextField'"

[arielelkin]

Hello! I'm trying to automate tests on a menu bar app, but the runner can't seem to find any user interface elements. For example, when trying to find a TextField it fails with

selenium.common.exceptions.NoSuchElementException: Message: unable to find an element using 'class name', value 'XCUIElementTypeTextField'

You can see a minimal reproducible example version of the app here: https://github.com/arielelkin/simplemenubarapp

Build it, drag and drop the binary into your /Applications directory, start the appium server (Appium v2.0.0-beta.55) run the following pytest:

import pytest

from appium import webdriver
# Options are available in Python client since v2.6.0
from appium.options.mac import Mac2Options
from appium.webdriver.common.appiumby import AppiumBy

@pytest.fixture()
def driver():
    options = Mac2Options()
    options.bundle_id = 'com.example.myapp'
    drv = webdriver.Remote('http://127.0.0.1:4723', options=options)
    yield drv
    drv.quit()

def test_edit_text(driver):
    edit_field = driver.find_element(by=AppiumBy.CLASS_NAME, value='XCUIElementTypeTextField')
    edit_field.send_keys('hello there')
    assert edit_field.text == 'hello there'
    edit_field.clear()
    assert edit_field.text == ''

Appium log

``` $ appium [Appium] Welcome to Appium v2.0.0-beta.55 [Appium] Attempting to load driver mac2... [debug] [Appium] Requiring driver at /Users/arielelkin/.appium/node_modules/appium-mac2-driver [Appium] Appium REST http interface listener started on 0.0.0.0:4723 [Appium] Available drivers: [Appium] - mac2@1.5.1 (automationName 'Mac2') [Appium] No plugins have been installed. Use the "appium plugin" command to install the one(s) you want to use. [debug] [HTTP] Request idempotency key: 8b167ca4-fc58-4f63-97a6-4ce5b8cdccde [HTTP] --> POST /session [HTTP] {"capabilities":{"firstMatch":[{}],"alwaysMatch":{"appium:automationName":"Mac2","platformName":"Mac","appium:bundleId":"com.example.myapp"}}} [debug] [AppiumDriver@e1b0] Calling AppiumDriver.createSession() with args: [null,null,{"firstMatch":[{}],"alwaysMatch":{"appium:automationName":"Mac2","platformName":"Mac","appium:bundleId":"com.example.myapp"}}] [debug] [AppiumDriver@e1b0] Event 'newSessionRequested' logged at 1676894704533 (12:05:04 GMT+0000 (Western European Standard Time)) [Appium] Attempting to find matching driver for automationName 'Mac2' and platformName 'Mac' [Appium] The 'mac2' driver was installed and matched caps. [Appium] Will require it at /Users/arielelkin/.appium/node_modules/appium-mac2-driver [debug] [Appium] Requiring driver at /Users/arielelkin/.appium/node_modules/appium-mac2-driver [AppiumDriver@e1b0] Appium v2.0.0-beta.55 creating new Mac2Driver (v1.5.1) session [AppiumDriver@e1b0] Checking BaseDriver versions for Appium and Mac2Driver [AppiumDriver@e1b0] Appium's BaseDriver version is 9.3.1 [AppiumDriver@e1b0] Mac2Driver's BaseDriver version is 9.3.1 [debug] [Mac2Driver@8994] Creating session with W3C capabilities: { [debug] [Mac2Driver@8994] "alwaysMatch": { [debug] [Mac2Driver@8994] "platformName": "Mac", [debug] [Mac2Driver@8994] "appium:automationName": "Mac2", [debug] [Mac2Driver@8994] "appium:bundleId": "com.example.myapp" [debug] [Mac2Driver@8994] }, [debug] [Mac2Driver@8994] "firstMatch": [ [debug] [Mac2Driver@8994] {} [debug] [Mac2Driver@8994] ] [debug] [Mac2Driver@8994] } [Mac2Driver@8994 (4aa5900c)] Session created with session id: 4aa5900c-3a74-471b-8bbe-bb0d960a1b9f [debug] [WebDriverAgentMac] Using bootstrap root: /Users/arielelkin/.appium/node_modules/appium-mac2-driver/WebDriverAgentMac [debug] [WebDriverAgentMac] Using xcodebuild binary at '/usr/bin/xcodebuild' [WebDriverAgentMac] WebDriverAgent sources are up to date (1676563483774 >= 1676563483774) [debug] [WebDriverAgentMac] Using 127.0.0.1 as server host [debug] [WebDriverAgentMac] Using port 10100 [WebDriverAgentMac] Mac2Driver host process logging is disabled. All the xcodebuild output is going to be suppressed. Set the 'showServerLogs' capability to 'true' if this is an undesired behavior [WebDriverAgentMac] Starting Mac2Driver host process: xcodebuild build-for-testing test-without-building -project /Users/arielelkin/.appium/node_modules/appium-mac2-driver/WebDriverAgentMac/WebDriverAgentMac.xcodeproj -scheme WebDriverAgentRunner COMPILER_INDEX_STORE_ENABLE\=NO [debug] [WD Proxy] Matched '/status' to command name 'getStatus' [debug] [WD Proxy] Proxying [GET /status] to [GET http://127.0.0.1:10100/status] with no body [WD Proxy] connect ECONNREFUSED 127.0.0.1:10100 [debug] [WD Proxy] Matched '/status' to command name 'getStatus' [debug] [WD Proxy] Proxying [GET /status] to [GET http://127.0.0.1:10100/status] with no body [WD Proxy] connect ECONNREFUSED 127.0.0.1:10100 [debug] [WD Proxy] Matched '/status' to command name 'getStatus' [debug] [WD Proxy] Proxying [GET /status] to [GET http://127.0.0.1:10100/status] with no body [WD Proxy] connect ECONNREFUSED 127.0.0.1:10100 [debug] [WD Proxy] Matched '/status' to command name 'getStatus' [debug] [WD Proxy] Proxying [GET /status] to [GET http://127.0.0.1:10100/status] with no body [debug] [WD Proxy] Got response with status 200: {"value":{"message":"WebDriverAgent is ready to accept commands","state":"success","os":{"version":"Version 13.2 (Build 22D49)"},"ready":true,"build":{"time":"Feb 16 2023 16:24:42"}},"sessionId":null} [WebDriverAgentMac] The host process is ready within 3180ms [debug] [WD Proxy] Matched '/session' to command name 'createSession' [debug] [WD Proxy] Proxying [POST /session] to [POST http://127.0.0.1:10100/session] with body: {"capabilities":{"firstMatch":[{}],"alwaysMatch":{"platformName":"Mac","automationName":"Mac2","bundleId":"com.example.myapp"}}} [debug] [WD Proxy] Got response with status 200: {"value":{"sessionId":"25A8ACC1-1014-4751-815C-67655BE3DA2B","capabilities":{"CFBundleIdentifier":"com.example.myapp"}},"sessionId":"25A8ACC1-1014-4751-815C-67655BE3DA2B"} [WD Proxy] Determined the downstream protocol as 'W3C' [AppiumDriver@e1b0] New Mac2Driver session created successfully, session 4aa5900c-3a74-471b-8bbe-bb0d960a1b9f added to master session list [debug] [AppiumDriver@e1b0] Event 'newSessionStarted' logged at 1676894708244 (12:05:08 GMT+0000 (Western European Standard Time)) [debug] [Mac2Driver@8994 (4aa5900c)] Cached the protocol value 'W3C' for the new session 4aa5900c-3a74-471b-8bbe-bb0d960a1b9f [debug] [Mac2Driver@8994 (4aa5900c)] Responding to client with driver.createSession() result: {"capabilities":{"platformName":"Mac","automationName":"Mac2","bundleId":"com.example.myapp"}} [HTTP] <-- POST /session 200 3716 ms - 155 [HTTP] [HTTP] --> POST /session/4aa5900c-3a74-471b-8bbe-bb0d960a1b9f/element [HTTP] {"using":"class name","value":"XCUIElementTypeTextView"} [debug] [Mac2Driver@8994 (4aa5900c)] Calling AppiumDriver.findElement() with args: ["class name","XCUIElementTypeTextView","4aa5900c-3a74-471b-8bbe-bb0d960a1b9f"] [debug] [Mac2Driver@8994 (4aa5900c)] Valid locator strategies for this request: id, name, accessibility id, xpath, class name, -ios predicate string, predicate string, -ios class chain, class chain [debug] [WD Proxy] Matched '/element' to command name 'findElement' [debug] [WD Proxy] Proxying [POST /element] to [POST http://127.0.0.1:10100/session/25A8ACC1-1014-4751-815C-67655BE3DA2B/element] with body: {"using":"class name","value":"XCUIElementTypeTextView"} [WD Proxy] Got response with status 404: {"value":{"error":"no such element","message":"unable to find an element using 'class name', value 'XCUIElementTypeTextView'","traceback":"(\n\t0 WebDriverAgentLib 0x000000010a38f272 FBNoSuchElementErrorResponseForRequest + 290\n\t1 WebDriverAgentLib 0x000000010a38effa +[FBFindElementCommands handleFindElement:] + 426\n\t2 WebDriverAgentLib 0x000000010a3d466c -[FBRoute_TargetAction mountRequest:intoResponse:] + 172\n\t3 WebDriverAgentLib 0x000000010a3ce4b8 __37-[FBWebServer registerRouteHandlers:]_block_invoke + 776\n\t4 WebDriverAgentLib 0x000000010a3c7bad -[RoutingHTTPServer handleRoute:withRequest:response:] + 189\n\t5 WebDriverAgentLib 0x000000010a3c88f1 __72-[RoutingHTTPServer routeMethod:withPath:parameters:request:connection:]_block_invoke + 65\n\t6 libdispatch.dylib 0x00007ff8069cda44 _dispatch_client_callout + 8\n\t7 libdispatch.dylib 0x000... [debug] [W3C] Matched W3C error code 'no such element' to NoSuchElementError [debug] [Mac2Driver@8994 (4aa5900c)] Encountered internal error running command: NoSuchElementError: unable to find an element using 'class name', value 'XCUIElementTypeTextView' [debug] [Mac2Driver@8994 (4aa5900c)] at errorFromW3CJsonCode (/usr/local/lib/node_modules/appium/node_modules/@appium/base-driver/lib/protocol/errors.js:1034:25) [debug] [Mac2Driver@8994 (4aa5900c)] at ProxyRequestError.getActualError (/usr/local/lib/node_modules/appium/node_modules/@appium/base-driver/lib/protocol/errors.js:909:14) [debug] [Mac2Driver@8994 (4aa5900c)] at WDAMacProxy.command (/usr/local/lib/node_modules/appium/node_modules/@appium/base-driver/lib/jsonwp-proxy/proxy.js:340:19) [debug] [Mac2Driver@8994 (4aa5900c)] at processTicksAndRejections (node:internal/process/task_queues:95:5) [debug] [Mac2Driver@8994 (4aa5900c)] at Mac2Driver.findElOrEls (/Users/arielelkin/.appium/node_modules/appium-mac2-driver/lib/commands/find.js:17:10) [debug] [Mac2Driver@8994 (4aa5900c)] at Mac2Driver.findElOrElsWithProcessing (/usr/local/lib/node_modules/appium/node_modules/@appium/base-driver/lib/basedriver/commands/find.js:81:16) [debug] [Mac2Driver@8994 (4aa5900c)] at Mac2Driver.findElement (/usr/local/lib/node_modules/appium/node_modules/@appium/base-driver/lib/basedriver/commands/find.js:21:14) [HTTP] <-- POST /session/4aa5900c-3a74-471b-8bbe-bb0d960a1b9f/element 404 158 ms - 1127 [HTTP] [HTTP] --> DELETE /session/4aa5900c-3a74-471b-8bbe-bb0d960a1b9f [HTTP] {} [debug] [Mac2Driver@8994 (4aa5900c)] Calling AppiumDriver.deleteSession() with args: ["4aa5900c-3a74-471b-8bbe-bb0d960a1b9f"] [debug] [AppiumDriver@e1b0] Event 'quitSessionRequested' logged at 1676894708492 (12:05:08 GMT+0000 (Western European Standard Time)) [AppiumDriver@e1b0] Removing session 4aa5900c-3a74-471b-8bbe-bb0d960a1b9f from our master session list [debug] [WD Proxy] Matched '/session/25A8ACC1-1014-4751-815C-67655BE3DA2B' to command name 'deleteSession' [debug] [WD Proxy] Proxying [DELETE /session/25A8ACC1-1014-4751-815C-67655BE3DA2B] to [DELETE http://127.0.0.1:10100/session/25A8ACC1-1014-4751-815C-67655BE3DA2B] with no body [debug] [WD Proxy] Got response with status 200: {"value":null,"sessionId":null} [debug] [AppiumDriver@e1b0] Event 'quitSessionFinished' logged at 1676894709524 (12:05:09 GMT+0000 (Western European Standard Time)) [debug] [AppiumDriver@e1b0] Received response: null [debug] [AppiumDriver@e1b0] But deleting session, so not returning [debug] [AppiumDriver@e1b0] Responding to client with driver.deleteSession() result: null [HTTP] <-- DELETE /session/4aa5900c-3a74-471b-8bbe-bb0d960a1b9f 200 1033 ms - 14 [HTTP] ```

Pytest log

``` $ tester.py ================================================================ test session starts ================================================================= platform darwin -- Python 3.10.8, pytest-7.2.1, pluggy-1.0.0 rootdir: /Users/arielelkin/Desktop collected 1 item tester.py F [100%] ====================================================================== FAILURES ====================================================================== ___________________________________________________________________ test_edit_text ___________________________________________________________________ driver = def test_edit_text(driver): > edit_field = driver.find_element(by=AppiumBy.CLASS_NAME, value='XCUIElementTypeTextView') tester.py:20: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ /usr/local/lib/python3.10/site-packages/appium/webdriver/webdriver.py:383: in find_element return self.execute(RemoteCommand.FIND_ELEMENT, {'using': by, 'value': value})['value'] /usr/local/lib/python3.10/site-packages/selenium/webdriver/remote/webdriver.py:440: in execute self.error_handler.check_response(response) /usr/local/lib/python3.10/site-packages/appium/webdriver/errorhandler.py:30: in check_response raise wde /usr/local/lib/python3.10/site-packages/appium/webdriver/errorhandler.py:26: in check_response super().check_response(response) _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = response = {'status': 404, 'value': '{"value":{"error":"no such element","message":"unable to find an element using \'class name\...lement (/usr/local/lib/node_modules/appium/node_modules/@appium/base-driver/lib/basedriver/commands/find.js:21:14)"}}'} def check_response(self, response: Dict[str, Any]) -> None: """Checks that a JSON response from the WebDriver does not have an error. :Args: - response - The JSON response from the WebDriver server as a dictionary object. :Raises: If the response contains an error message. """ status = response.get("status", None) if not status or status == ErrorCode.SUCCESS: return value = None message = response.get("message", "") screen: str = response.get("screen", "") stacktrace = None if isinstance(status, int): value_json = response.get("value", None) if value_json and isinstance(value_json, str): import json try: value = json.loads(value_json) if len(value.keys()) == 1: value = value["value"] status = value.get("error", None) if not status: status = value.get("status", ErrorCode.UNKNOWN_ERROR) message = value.get("value") or value.get("message") if not isinstance(message, str): value = message message = message.get("message") else: message = value.get("message", None) except ValueError: pass exception_class: Type[WebDriverException] if status in ErrorCode.NO_SUCH_ELEMENT: exception_class = NoSuchElementException elif status in ErrorCode.NO_SUCH_FRAME: exception_class = NoSuchFrameException elif status in ErrorCode.NO_SUCH_SHADOW_ROOT: exception_class = NoSuchShadowRootException elif status in ErrorCode.NO_SUCH_WINDOW: exception_class = NoSuchWindowException elif status in ErrorCode.STALE_ELEMENT_REFERENCE: exception_class = StaleElementReferenceException elif status in ErrorCode.ELEMENT_NOT_VISIBLE: exception_class = ElementNotVisibleException elif status in ErrorCode.INVALID_ELEMENT_STATE: exception_class = InvalidElementStateException elif ( status in ErrorCode.INVALID_SELECTOR or status in ErrorCode.INVALID_XPATH_SELECTOR or status in ErrorCode.INVALID_XPATH_SELECTOR_RETURN_TYPER ): exception_class = InvalidSelectorException elif status in ErrorCode.ELEMENT_IS_NOT_SELECTABLE: exception_class = ElementNotSelectableException elif status in ErrorCode.ELEMENT_NOT_INTERACTABLE: exception_class = ElementNotInteractableException elif status in ErrorCode.INVALID_COOKIE_DOMAIN: exception_class = InvalidCookieDomainException elif status in ErrorCode.UNABLE_TO_SET_COOKIE: exception_class = UnableToSetCookieException elif status in ErrorCode.TIMEOUT: exception_class = TimeoutException elif status in ErrorCode.SCRIPT_TIMEOUT: exception_class = TimeoutException elif status in ErrorCode.UNKNOWN_ERROR: exception_class = WebDriverException elif status in ErrorCode.UNEXPECTED_ALERT_OPEN: exception_class = UnexpectedAlertPresentException elif status in ErrorCode.NO_ALERT_OPEN: exception_class = NoAlertPresentException elif status in ErrorCode.IME_NOT_AVAILABLE: exception_class = ImeNotAvailableException elif status in ErrorCode.IME_ENGINE_ACTIVATION_FAILED: exception_class = ImeActivationFailedException elif status in ErrorCode.MOVE_TARGET_OUT_OF_BOUNDS: exception_class = MoveTargetOutOfBoundsException elif status in ErrorCode.JAVASCRIPT_ERROR: exception_class = JavascriptException elif status in ErrorCode.SESSION_NOT_CREATED: exception_class = SessionNotCreatedException elif status in ErrorCode.INVALID_ARGUMENT: exception_class = InvalidArgumentException elif status in ErrorCode.NO_SUCH_COOKIE: exception_class = NoSuchCookieException elif status in ErrorCode.UNABLE_TO_CAPTURE_SCREEN: exception_class = ScreenshotException elif status in ErrorCode.ELEMENT_CLICK_INTERCEPTED: exception_class = ElementClickInterceptedException elif status in ErrorCode.INSECURE_CERTIFICATE: exception_class = InsecureCertificateException elif status in ErrorCode.INVALID_COORDINATES: exception_class = InvalidCoordinatesException elif status in ErrorCode.INVALID_SESSION_ID: exception_class = InvalidSessionIdException elif status in ErrorCode.UNKNOWN_METHOD: exception_class = UnknownMethodException else: exception_class = WebDriverException if not value: value = response["value"] if isinstance(value, str): raise exception_class(value) if message == "" and "message" in value: message = value["message"] screen = None # type: ignore[assignment] if "screen" in value: screen = value["screen"] stacktrace = None st_value = value.get("stackTrace") or value.get("stacktrace") if st_value: if isinstance(st_value, str): stacktrace = st_value.split("\n") else: stacktrace = [] try: for frame in st_value: line = frame.get("lineNumber", "") file = frame.get("fileName", "") if line: file = f"{file}:{line}" meth = frame.get("methodName", "") if "className" in frame: meth = f"{frame['className']}.{meth}" msg = " at %s (%s)" msg = msg % (meth, file) stacktrace.append(msg) except TypeError: pass if exception_class == UnexpectedAlertPresentException: alert_text = None if "data" in value: alert_text = value["data"].get("text") elif "alert" in value: alert_text = value["alert"].get("text") raise exception_class(message, screen, stacktrace, alert_text) # type: ignore[call-arg] # mypy is not smart enough here > raise exception_class(message, screen, stacktrace) E selenium.common.exceptions.NoSuchElementException: Message: unable to find an element using 'class name', value 'XCUIElementTypeTextView' E Stacktrace: E NoSuchElementError: unable to find an element using 'class name', value 'XCUIElementTypeTextView' E at errorFromW3CJsonCode (/usr/local/lib/node_modules/appium/node_modules/@appium/base-driver/lib/protocol/errors.js:1034:25) E at ProxyRequestError.getActualError (/usr/local/lib/node_modules/appium/node_modules/@appium/base-driver/lib/protocol/errors.js:909:14) E at WDAMacProxy.command (/usr/local/lib/node_modules/appium/node_modules/@appium/base-driver/lib/jsonwp-proxy/proxy.js:340:19) E at processTicksAndRejections (node:internal/process/task_queues:95:5) E at Mac2Driver.findElOrEls (/Users/arielelkin/.appium/node_modules/appium-mac2-driver/lib/commands/find.js:17:10) E at Mac2Driver.findElOrElsWithProcessing (/usr/local/lib/node_modules/appium/node_modules/@appium/base-driver/lib/basedriver/commands/find.js:81:16) E at Mac2Driver.findElement (/usr/local/lib/node_modules/appium/node_modules/@appium/base-driver/lib/basedriver/commands/find.js:21:14) /usr/local/lib/python3.10/site-packages/selenium/webdriver/remote/errorhandler.py:245: NoSuchElementException ============================================================== short test summary info =============================================================== FAILED tester.py::test_edit_text - selenium.common.exceptions.NoSuchElementException: Message: unable to find an element using 'class name', value 'XCUIElementTypeTextView' ================================================================= 1 failed in 5.12s ================================================================== ```

[mykola-mokhnach]

is the element present in the page source?

[arielelkin]

Could you please clarify what you refer by "page source"? This is a macOS app... The UI element shows up in accessibility inspector:

[mykola-mokhnach]

i mean the output of https://www.w3.org/TR/webdriver/#dfn-get-page-source API

[arielelkin]

I've looked through the output of driver.page_source and could not find any text field. Not sure why that is..? It's showing up in the Accessibility Inspector...

[mykola-mokhnach]

I also don't know why. Perhaps, there is an issue in XCTest itself...

[mykola-mokhnach]

Or maybe this particular dialog does belong to a different application. You could try https://github.com/appium/appium-mac2-driver#application-under-test-concept to verify that

[mykola-mokhnach]

Also possible duplicate of https://github.com/appium/appium-mac2-driver/issues/178

[arielelkin]

I think the "application under test" is indeed the app I'm testing. I can see the test launching as expected, and it doesn't terminate it.

I believe it's premature to label this as "not an issue", especially since XCUITest has been around for quite a while and I know it's able to handle text fields properly (I know it is if I run UI tests from Xcode)...

Are you able to reproduce the problem with the sample application I linked to above?

[mykola-mokhnach]

I think the "application under test" is indeed the app I'm testing. I can see the test launching as expected, and it doesn't terminate it.

Have you tried to switch to other apps being active at the moment? Are you sure this particular dialog belongs to your app in the accessibility tree and not Finder, for example?

[arielelkin]

Have you tried to switch to other apps being active at the moment?

Yes

Are you sure this particular dialog belongs to your app in the accessibility tree and not Finder, for example?

Yes. In fact you can see the placeholder text in the app matches the placeholder text shown in the Accessibility Inspector.

Feel free to reproduce the issue with the sample application I linked to above...

[arielelkin]

I can confirm that this issue is happening for regular floating window apps too, not just menu bar apps.

Here's a small sample app where the test above fails with the same issue:

https://github.com/arielelkin/macostestapp