galenframework / galen

Layout and functional testing framework for websites
http://galenframework.com
1.41k stars 163 forks source link

Problem with mobile testing #466

Open trignolo opened 7 years ago

trignolo commented 7 years ago

I'm currently try to test some web pages using Galen (I want to introduce this tool as a standard in my company, if possible), but I've some problems with mobile testing.

1) Local testing For local testing I'm using jsfactory to instantiate chrome, here the code: `importClass(org.openqa.selenium.chrome.ChromeOptions); importClass(org.openqa.selenium.chrome.ChromeDriver); importClass(org.openqa.selenium.remote.DesiredCapabilities); importClass(com.galenframework.utils.GalenUtils); importClass(com.galenframework.browser.SeleniumBrowser);

var pageUrl = args[0]; var size = GalenUtils.readSize(args[1]);

options = new ChromeOptions();

options.addArguments("--user-agent=Mozilla/5.0 (Linux; Android 4.4.2; Nexus 5 Build/KOT49H) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/32.0.1700.99 Mobile Safari/537.36"); capabilities = DesiredCapabilities.chrome(); capabilities.setCapability(ChromeOptions.CAPABILITY, options); var browser = new SeleniumBrowser(new ChromeDriver(capabilities));

browser.load(pageUrl); browser.changeWindowSize(size);

browser;`

I've also configured galen.browser.pageElement.areaFinder=jsbased_native in galen config, to improve the element position detection on the page.

When I run the test on desktop enviroment all is working properly, but when I run test for mobile devices I've problem with resulting reports. This is the main portion of the galen suite useful for mobile testing:

` @@ set base_url

@@ table devicesMobile | deviceName | tags | size | | Nexus5 | mobile | 360x640 |

@@ groups mobile @@ parameterized using devicesMobile Home page on ${deviceName} device jsfactory ${GALEN_TEST_ROOT}/jsfactory/chrome-driver.js ${base_url} ${size} cookie "__ric=prova; path=/" wait 5s check ${GALEN_TEST_ROOT}/modules/big-cards/big-cards.gspec --include "${tags}"`

In the final report I can see many errors, but the strange thing is the resulting screenshot and position detection of element in the page. Here an example:

galen-mobile-dimension-problem

It seems that the image is bigger than what it should be, considering that the device width is set to 360px.

I attach also the complete galen report. galen-report-mobile.zip

1) Browserstack testing Testing on browserstack has a lilttle bit different problem, in this case it seems that the vertical position of the element in page is affected of some form of extra pixels, added on top of the page. Here a screenshot i which you can see problem: the pink rectangle sottotitolo is located above its real position (it's the text that starts with Nella fabbrica della Rover...).

galen-browserstack-problem

My rough idea is that in browserstack the screenshot include also the address and tab bar of the device and this beaviour cause the extra pixels. Here the complete galen report for BS test:

galen-report-mobile-BS.zip

Thanks in advance

jfernandogt commented 7 years ago

I have the same issue.

goatsy commented 7 years ago

I have the same issue when testing with emulators. Browserstack now also offers real devices for automated testing (which I experienced are faster) and I also got rid of the problem with the heatmaps.

jfernandogt commented 7 years ago

I solved it:

  1. add galen config file to the main path of the project.
  2. change the fullpage config to true

Note: In some cases if you use java and testng you need load the page and wait some seconds to test it.