search

percy / percy-ember

Ember addon for visual regression testing with Percy
https://docs.percy.io/docs/ember
MIT License
73 stars 44 forks source link

percy build only hangs #1

Closed kellyselden closed 8 years ago

kellyselden commented 8 years ago

Trying to add percy to try it out here

Could be because I'm using ember-exam to parallelize tests across travis jobs.

fotinakis commented 8 years ago

Hey @kellyselden — yeah the parallelization is definitely the issue, this is where it gets slightly complicated.

Can you try adding a new env var called PERCY_PARALLEL_TOTAL? It should be set to the number of partitions/nodes in your parallel setup.

There's some info on https://percy.io/docs/learn/parallel-tests (you can ignore the "nonce", it should be set already if you're on Travis).

Let me know if that works!

kellyselden commented 8 years ago

I added that var, and although now it has collapsed the percy builds into one, it is still hanging on receiving. I'm unsure if I should keep waiting or if it really hung.

fotinakis commented 8 years ago

Hmm. So you have 5 partitions in exam — do you know if that translates into 5 separate node processes + browsers + testem runners for your testing setup? If there was 1 node process, but 5 test runners I could see that bug happening (the ember-percy addon sends a finalize command at the end of each test suite run, if you have 5 of them, the backend will expect 5 finalize calls underneath).

fotinakis commented 8 years ago

Ahhh you are also using pretender which has the same problems as described in the Mirage troubleshooting (blocks internal percy requests by default).

This is untested code, but can you try adding this to your pretender config?

this.post('/_percy/:method', this.passthrough);
kellyselden commented 8 years ago

I pushed that change, and it caused a different test not using percy to fail. Not sure where to start digging.

fotinakis commented 8 years ago

What is the error?

kellyselden commented 8 years ago

No error, just a test failure "identical calls to github api are consolidated", which sounds like the pretender change had more impact that anticipated.

fotinakis commented 8 years ago

It looks like that error is this line in your code: https://github.com/kellyselden/package-hint-historic-resolver/blob/837fe0d796e8296a21457bf0ac030327ce4d223a/tests/acceptance/application-test.js#L138 ... which makes total sense because server.handlers[0].numberOfCalls is now going to be larger than 1 because of the percy calls. Can you think of a different way to test that API calls are consolidated?

fotinakis commented 8 years ago

Or maybe it is as simple as just server.handlers[1]?

kellyselden commented 8 years ago

Wow thanks for taking the time to dig in. I didn't have time at the time. I made that change and the build passes, percy generates the image, but is still receiving.

fotinakis commented 8 years ago

Can you try unsetting the PERCY_PARALLEL_TOTAL env var? Percy is only receiving 1 finalize call (not 5) from your tests, which leads me to believe there is only one node process in your environment. Removing the var will fix it temporarily. However, this might only be a short-term fix — if you add percy snapshots to multiple places and some of them don't show up, we may need to dig into ember-exam compatibility more thoroughly.