kverma20
commented
4 years ago 
The attached screenshot is taken by the plugin and saved at /usr/local/lib/node_modules/test-ai-classifier/build-js/lib
Desired Capabilities used - HashMap<String, String> customFindModules = new HashMap<>(); customFindModules.put("ai", "test-ai-classifier");
capabilities.setCapability("customFindModules", customFindModules);
capabilities.setCapability("testaiFindMode", "object_detection");
capabilities.setCapability("testaiObjectDetectionThreshold", 0.2);
capabilities.setCapability("shouldUseCompactResponses", false);Tried below functions to find the element in test case - getAppDriver().findElementByCustom("search").click(); getAppDriver().findElement("-custom","ai:search").click(); getAppDriver().findElement(MobileBy.custom("ai:search")).click();
jlipps
The appium classifier plugin is not able to find the element. I could see there are few information related to model, but not sure if that's the root cause of the issue.
`[HTTP] --> POST /wd/hub/session/0233bd55-e62c-49c4-aac8-912134a17d58/element [HTTP] {"using":"-custom","value":"search"} [debug] [W3C (0233bd55)] Calling AppiumDriver.findElement() with args: ["-custom","search","0233bd55-e62c-49c4-aac8-912134a17d58"] [debug] [BaseDriver] Find plugin 'ai' requested; will attempt to use it from 'test-ai-classifier' 2019-12-23 14:30:45.216136: I tensorflow/core/platform/cpu_feature_guard.cc:142] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA 2019-12-23 14:30:45.241688: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x11d1e3290 initialized for platform Host (this does not guarantee that XLA will be used). Devices: 2019-12-23 14:30:45.241721: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): Host, Default Version [debug] [BaseDriver] Waiting up to 0 ms for condition [ai] Will use object detection method for finding elements via AI [ai] Setting confidence threshold to default value of 0.2 [ai] Getting window size in case we need to scale screenshot [debug] [WD Proxy] Matched '/window/current/size' to command name 'getWindowSize' [debug] [WD Proxy] Proxying [GET /window/current/size] to [GET http://localhost:8207/wd/hub/session/087c93f3-8cdb-42cf-8a10-3294cac360b7/window/current/size] with body: {} [debug] [WD Proxy] Got response with status 200: "{\"sessionId\":\"087c93f3-8cdb-42cf-8a10-3294cac360b7\",\"status\":0,\"value\":{\"height\":1504,\"width\":2560}}" [ai] Getting screenshot to use for classifier [debug] [WD Proxy] Matched '/screenshot' to command name 'getScreenshot' [debug] [WD Proxy] Proxying [GET /screenshot] to [GET http://localhost:8207/wd/hub/session/087c93f3-8cdb-42cf-8a10-3294cac360b7/screenshot] with no body [debug] [WD Proxy] Got response with status 200: "{\"sessionId\":\"087c93f3-8cdb-42cf-8a10-3294cac360b7\",\"status\":0,\"value\":\"iVBORw0KGgoAAAANSUhEUgAACgAAAAZACAYAAAC1rBvAAAAABHNCSVQICAgIfAhkiAAAIABJREFU\neJzUfXm8XVV1\/\/e+l\/llDhkIZGIMJEAAIQjKEKuCA40TxAmoraK\/CmpVKpZWKihWrAO0tWhRwSmo\nFRApDhRwYHJg0CQyJzEBMpCQl\/Elee+d3x+PczlvZ69x73Nfuvz4efees\/daa6+9hu\/aOdzTWHjp\nxAJKaqDR\/FygaF4rP3PzpDFW8vKszis\/16FfTN7ewEfiF16PjePsVvqIxEOjG8fLwpNbQ06ibBf6\nHIBs9pB08FKdMeGVm+seZU9pzXurbXPmMUtsa3NIjnuWcZZ4SZWVyjOHjl7ZKeNiMQXo12HVI0Xv\nnHuY4r8D4Y9emSk5JaXG5aBUbGjRnxpbl39rKRc20\/o79ZmTrZHv1Se8Zq31qfLr4F8X1VUTUmTn\nxDJ7I6X2L57xrbSppw57bELltPCahTzYJ9Ql\/Bz7m0Ix\/qmkPcspSStbc6YQ3qurF8vF17KmVF5a\nf65rbanjU+KRq6m5sZMlZ1mxYm6b5iKrH0sYS2sTT97w8NBQq3vwUqanr9Xsl9W2KbiVqnOxa56+\nmesbqOueazmwu5a8ucCa\/7w910DEg0TWnlxjJ69Ppujrjbu6+06NHAsPoH+vXf1ula2pwdYcWurj\nqfk5KCdepO55\/KIVebCVdb3VeDEnj70hljmeKfZMxQVenTXfNXNz9n9avaVxKT2GZW4dvUxuTCKN\nz23bVvVrOWRKfZq1t9Xol7uutGpfvP2VhWeu2qzBN7mwlEVXi+7SXKvsVuwfp3POteQcW45vP+zU\nkZeqZwSTcyoV41e9ppU30OTVU5oX3k+Ro0kCGj570w... [ai] Turning screenshot into HTML image for use with canvas [ai] Writing screenshot to /usr/local/lib/node_modules/test-ai-classifier/build-js/lib/Mon, 23 Dec 2019 09:00:46 GMT_classifiertest.png for use in object detection [ai] Gathering object candidates from tensorflow native 2019-12-23 14:30:46.785228: I tensorflow/cc/saved_model/reader.cc:31] Reading SavedModel from: /usr/local/lib/node_modules/test-ai-classifier/model/obj_detection_model 2019-12-23 14:30:46.950439: I tensorflow/cc/saved_model/reader.cc:54] Reading meta graph with tags { serve } 2019-12-23 14:30:47.595144: I tensorflow/cc/saved_model/loader.cc:202] Restoring SavedModel bundle. 2019-12-23 14:30:47.595208: I tensorflow/cc/saved_model/loader.cc:212] The specified SavedModel has no variables; no checkpoints were restored. File does not exist: /usr/local/lib/node_modules/test-ai-classifier/model/obj_detection_model/variables/variables.index 2019-12-23 14:30:47.595237: I tensorflow/cc/saved_model/loader.cc:311] SavedModel load for tags { serve }; Status: success. Took 810005 microseconds. [ai] Candidates retrieved [ai] Making label predictions based on element images This model execution did not contain any nodes with control flow or dynamic output shapes. You can use model.execute() instead. [ai] Predictions for 7 element(s) took 0.398 seconds [ai] Found 0 matching elements [ai] Registering image elements with driver [debug] [W3C (0233bd55)] Encountered internal error running command: NoSuchElementError: An element could not be located on the page using the given search parameters. [debug] [W3C (0233bd55)] at AndroidUiautomator2Driver.findByCustom (/usr/local/lib/node_modules/appium/node_modules/appium-base-driver/lib/basedriver/commands/find.js:176:13) [HTTP] <-- POST /wd/hub/session/0233bd55-e62c-49c4-aac8-912134a17d58/element 404 17981 ms - 397 [HTTP]
org.openqa.selenium.NoSuchElementException: An element could not be located on the page using the given search parameters. For documentation on this error, please visit: https://www.seleniumhq.org/exceptions/no_such_element.html Build info: version: '3.141.59', revision: 'e82be7d358', time: '2018-11-14T08:17:03' System info: host: 'm-c02xf25rjg5h', ip: 'fe80:0:0:0:4fb:1360:f4af:594b%en0', os.name: 'Mac OS X', os.arch: 'x86_64', os.version: '10.14.6', java.version: '1.8.0_192' Driver info: io.appium.java_client.android.AndroidDriver Capabilities {app: /Users/kverma1/Documents/mo..., appPackage: com.asda.android, appWaitActivity: com.asda.android., automationName: UIAutomator2, avd: Nexus_10_API_27, avdLaunchTimeout: 300000, avdReadyTimeout: 300000, browserName: , customFindModules: {ai: test-ai-classifier}, databaseEnabled: false, desired: {app: /Users/kverma1/Documents/mo..., appWaitActivity: com.asda.android., automationName: UIAutomator2, avd: Nexus_10_API_27, avdLaunchTimeout: 300000, avdReadyTimeout: 300000, browserName: , customFindModules: {ai: test-ai-classifier}, deviceName: Nexus_10_API_27, fullReset: false, newCommandTimeout: 120, noReset: true, platformName: android, platformVersion: 8.1, shouldUseCompactResponses: false, systemPort: 8207, testaiFindMode: object_detection, testaiObjectDetectionThreshold: 0.3, uiautomator2ServerLaunchTimeout: 300000}, deviceApiLevel: 27, deviceManufacturer: Google, deviceModel: Android SDK built for x86, deviceName: emulator-5554, deviceScreenDensity: 320, deviceScreenSize: 2560x1600, deviceUDID: emulator-5554, fullReset: false, javascriptEnabled: true, locationContextEnabled: false, networkConnectionEnabled: true, newCommandTimeout: 120, noReset: true, pixelRatio: 2, platform: LINUX, platformName: Android, platformVersion: 8.1.0, shouldUseCompactResponses: false, statBarHeight: 48, systemPort: 8207, takesScreenshot: true, testaiFindMode: object_detection, testaiObjectDetectionThreshold: 0.3, uiautomator2ServerLaunchTimeout: 300000, viewportRect: {height: 1456, left: 0, top: 48, width: 2560}, warnings: {}, webStorageEnabled: false} Session ID: 0233bd55-e62c-49c4-aac8-912134a17d58 *** Element info: {Using=-custom, value=search}