Autodesk / orion-ui

UI Components that work seamlessly in multiple UI frameworks
https://goo.gl/shxHPB
Other
22 stars 6 forks source link

Visual regression testing should run against Safari #172

Open camwest opened 7 years ago

camwest commented 7 years ago

Please walk through the following steps for reporting an issue about Orion:

Short description of your issue:

Currently we only do visual regression testing on Firefox, Edge, and Chrome. Let's add Safari into the mix

How do we reproduce the issue?

N/A

What browsers are affected?

N/A

Which Orion version is affected?

N/A

aneyzberg commented 7 years ago

@camwest this is still an Issue on Sauce labs end. I sent another follow up email with regards to the previous ticket to see what the status is.

aneyzberg commented 7 years ago

The exception that we were seeing initially that was preventing us from implementing safari was the following - 13:56:51.035 WARN - Exception: Unknown command: {"id":"lr4c84nz93gg","name":"mouseMoveTo","parameters":{"yoffset":0,"xoffset":0,"element":":wdc:1489611411016"}} (WARNING: The server did not provide any stacktrace information)

The Sauce labs people said this was a selenium bug - and they said the following -

We've recently added support for the newly released Selenium 3.0.1, 
as least for part of our cloud. 
 For the latest version of Safari only (Safari 10 on OS X 10.12), 
you can use the new, built in Selenium support which _does_ allow for mouse movements.

To use Selenium 3.0.1, you'll need to add the 'seleniumVersion' 
desired capability to your Safari tests, setting it to '3.0.1'.

So I added this to our gemini.yml

  safari:
    desiredCapabilities:
      browserName: 'Safari'
      version: '10'
      platform: 'OS X 10.12'
      name: 'gemini visual regression tests'
      seleniumVersion: '3.0.1'

However this support appears to be inconsistent. And that exception is still thrown. However the main issue is it seems that the line intersections in the graph are what are failing the build as of now

safari diff

So this issue needs to be resolved