search

thaliproject / postcardapp

A sample app to demonstrate how to build Thali applications
MIT License
22 stars 5 forks source link

PostcardApp E2E UX tests #80

Closed deadlyfingers closed 8 years ago

deadlyfingers commented 8 years ago

End to end testing for Polymer web app

deadlyfingers commented 8 years ago

Looking at E2E potential test setups for Polymer Cordova/Node projects -

Automation for web apps: http://appium.io/

Travis-CI and SauceLabs are well used examples but wondering if they can be configured for jxcore app?

Related documentation: https://www.npmjs.com/package/web-component-tester https://github.com/Polymer/tools/tree/master/travis

deadlyfingers commented 8 years ago

Trying out Appium node code-samples but hitting environment error:

Error: [init({"browserName":"","appium-version":"1.3","platformName":"iOS","platformVersion":"8.1","deviceName":"iPhone Simulator","app":"http://appium.github.io/appium/assets/TestApp7.1.app.zip"})] The environment you requested was unavailable.

Seems Appium is set to iOS 8.1 by default, but new versions of Xcode are on 9.2 I updated an issue with a quick fix for running on iOS 9.2 - https://github.com/appium/sample-code/issues/55

And if you use the Appium app then you run into more config issues: https://github.com/appium/appium/issues/5705 I got it to run by modifying iOS Settings to use Platform Version "9.2" and checking "Use Native Instruments Library" in iOS Advanced Settings.

deadlyfingers commented 8 years ago

Getting it to work ok on emulators but problems hooking into webview on devices.

Android device:

Chromedriver: Error: An unknown server-side error occurred while processing the command. (Original error: unknown error: Chrome version must be >= 43.0.2357.0

iOS device: Can't seem to get the webview context in app.

deadlyfingers commented 8 years ago

Debugging webview on iOS Device requires additional setup:

  1. Install brew install ios-webkit-debug-proxy https://github.com/appium/appium/blob/master/docs/en/advanced-concepts/ios-webkit-debug-proxy.md
  2. Start appium and proxy using device's UDID: ios_webkit_debug_proxy -c DEVICE_UDID:27753-27754 -d
deadlyfingers commented 8 years ago

Got appium and mocha running tests with Postcard app, but having some trouble getting the correct selectors to write tests for express app running inside an iframe...

deadlyfingers commented 8 years ago

iframe could be a blocking issue with using Appium (otherwise would have been great to use): https://github.com/appium/appium/issues/5560 Doesn't appear to support switch to iframe... using driver.switchTo().frame("frameId") or driver.frame("frameId")

Error: [frame("appFrame")] Error response status: 8, NoSuchFrame - A request to switch to a frame could not be satisfied because the frame could not be found. Selenium error: A request to switch to a frame could not be satisfied because the frame could not be found.
deadlyfingers commented 8 years ago

Will see if postcard app can run ok without the need of an iframe - issue #117.

deadlyfingers commented 8 years ago

Appium hybrid app UX tests are working after removing need for iframe.

deadlyfingers commented 8 years ago

Issue with dismissing soft keyboard ('Done' on iOS) with textarea. Related discussion: https://discuss.appium.io/t/we-use-appium-1-2-4-with-java-node-we-need-to-click-on-soft-keyboards-done-action-is-there-any-way-to-achive-this/1380/14 https://github.com/appium/appium/issues/173 https://github.com/appium/appium/issues/4013

Also a related issue is getting value of textarea: http://stackoverflow.com/questions/13990566/how-to-get-entered-text-from-a-textbox-in-selenium I find using safeEval to execute javascript to get textarea value works.

return driver
      .waitForElementByCss("textarea", asserters.isDisplayed, defaults.wait.short)
      .waitForConditionInBrowser('document.querySelector("textarea").value.length > 0', 5000)
      .safeEval('document.querySelector("textarea").value')
      .should.eventually.contain(message);

Another issue is selecting (Polymer) custom elements with web driver: http://stackoverflow.com/questions/13702243/how-to-locate-custom-html-elements-using-selenium-webdriver

deadlyfingers commented 8 years ago

Found a slight gotya to be aware of when writing tests... When deleting an item (postcard) from an iron-list element I was getting an AssertionError: on the number of rows or items remaining. I found out it's because Polymer does not update the DOM tree and just hides the element (which I presume is for speed reasons). 

safeEval('document.querySelectorAll("#allPostcards .row:not([hidden])").length')
deadlyfingers commented 8 years ago

A feature that might be nice is taking screenshots when any tests fail for reporting. But takeScreenshot() doesn't seem to be able to output an image (using iOS simulator) http://stackoverflow.com/questions/15902142/correct-syntax-for-taking-screenshots-with-seleniums-webdriverjs-on-node

deadlyfingers commented 8 years ago

Todo: Need to fix formatting on github readme.md as the numbered lists aren't being recognised properly after using a code block.