percy / percy-ember

Ember addon for visual regression testing with Percy
https://docs.percy.io/docs/ember
MIT License
73 stars 44 forks source link

CSS is applied in one test but not the other #332

Closed twokul closed 3 years ago

twokul commented 3 years ago

This screenshot appears to be artificially darker in colour than it is supposed to be. When we compare it to a screenshot generated with our previous tool, BackstopJS, you can see that the Percy screenshot is noticeably darker: the font colour is closer to black than grey, and even the border around the search input, and the magnifying glass icon, are all darker than they are in the backstop image.

I have also reached out to support@ and provided the link to the build. Posting here, as well. I have also provided the screenshots since I cannot share them publicly.

Our stack is Ember + QUnit. @percy/cli is "^1.0.0-beta.54" and @percy/ember is "^3.0.0".

Let me know if there is any other information I can provide.

Robdel12 commented 3 years ago

Hey @twokul! The title suggests that there are snapshots with missing resources/assets. If so, it'd be great to have a link to a broken & also a working snapshot.

But, this makes me think otherwise:

When we compare it to a screenshot generated with our previous tool, BackstopJS, you can see that the Percy screenshot is noticeably darker

This is likely due to the difference of OS's and machines that screenshots are captured on. We're capturing screenshots in ubuntu (for Chrome & Firefox. Edge renders in windows, I believe -- double checking with the infra folks), which won't render the same as a macOS, windows, or various flavors of linux.

twokul commented 3 years ago

@Robdel12 Here is what we see:

You are correct to say that macOS, Windows and Linux have differences when it comes to some of the HTML controls (<input type="range">, for example) and font rendering. I am not sure if this is the case here, though.

The title suggests that there are snapshots with missing resources/assets. If so, it'd be great to have a link to a broken & also a working snapshot.

I do not think I can share the screenshots publicly, unfortunately. I did provide them in the email thread when I reached out to support@percy.io.

Robdel12 commented 3 years ago

Gotcha, I pulled up the ticket (I don't work support anymore, but I'll help out here). I'm confused as to what's going on/what the ask is. Are you looking to get Percy to match what Backstop renders? Or are you seeing Percy renders flip/change (like there are diffs in the snapshot)?

I haven't spotted any diffs in snapshots where fonts/things aren't loading (or flapping diffs in your builds). The linked snapshots in the email chain don't have any diffs. It seems like the ask is to get Percy to match what Backstop is outputting? This appears to be a difference between where the screenshot is captured. Percy's will be different from backstop, but it will be consistent (the machines/os/browser doesn't change)

twokul commented 3 years ago

Gotcha, I pulled up the ticket (I don't work support anymore, but I'll help out here)

Thanks much, I appreciate that! 🤝

I'm confused as to what's going on/what the ask is.

As I mentioned in my previous comment: we see the CSS applied in one test but not the other. The fonts load correctly, indeed, but the CSS is not applied correctly (particularly, the colour of the text, grey vs. black.)

The confusing bit to me is how can the styles work in one test and not the other when they "boot up" the same application with the same set of styles? 🤔

If you open this build:

Same application, same route 😬

It seems like the ask is to get Percy to match what Backstop is outputting?

Not at all, that would be unreasonable, imo 😄 The backstop screenshot was provided only to demonstrate how the page is supposed to look when everything is captured correctly.

github-actions[bot] commented 3 years ago

This issue is stale because it has been open for more than 14 days with no activity. Remove stale label or comment or this will be closed in 14 days.

github-actions[bot] commented 3 years ago

This issue was closed because it has been stalled for 28 days with no activity.