search

chenglou / node-huxley

Codeless front-end testing.
MIT License
361 stars 29 forks source link

Add option to save image when different #49

Closed bzuillsmith closed 10 years ago

bzuillsmith commented 10 years ago

I want to easily check the differences between expected and actual screenshot images. I think a good first step is to have an option to save the diff image(s). Then I could use a visual diff tool to see what changed.

chenglou commented 10 years ago

Not sure what you mean; if you run hux (defaults to compare mode), then you do have a diff.png saved when there are mismatch. Check your screenshot folder.

bzuillsmith commented 10 years ago

Hmmm, then I must be having other issues. I ran hux, and it did not create a diff.png. It just ran, but kept firefox open, and stopped running the process without any useful output to the console. I didn't know what the expected behavior was since it's not in the readme documentation.

Running this in Windows.

chenglou commented 10 years ago

Well first, are you in record mode or playback mode? If you're in playback, if the browser doesn't close then that's a bug. Can you repro it? I'll test it out.

bzuillsmith commented 10 years ago

I recorded with the <h1> text as text and then changed it to text1 and it froze up on playback. It finishes successfully if there are no changes to index.html (as expected). Only stops if there is a difference.

My Huxleyfile.json:

[{
    "name": "index",
    "url": "http://localhost:8080/index.html"
}]

My index.html:

<html>
<head></head>
<body>
<h1>
    test
</h1>
</body>
</html>

My console output (from git bash, but same thing happens in Windows cmd prompt):

Ben@BEN-PC ~/Desktop/blarg
$ selenium & hux -r
[1] 3524

firefox opening.

At index.hux
Selenium is started.
All output can be found at: c:\Users\Ben\AppData\Roaming\npm\node_modules\seleni
um-server\lib\runner/../../logs
Begin record
Type q to quit, l for taking a screenshot and marking a live playback point til
next screenshot, and anything else to take a normal screenshot.
>
1 screenshot recorded.
> q

Don't move! Simulating the recording now...

firefox opening.

At index.hux
  Taking screenshot firefox-1.png

All done successfully!
[1]+  Done                    selenium

Ben@BEN-PC ~/Desktop/blarg
$ selenium & hux
[1] 8028
Selenium is started.
All output can be found at: c:\Users\Ben\AppData\Roaming\npm\node_modules\seleni
um-server\lib\runner/../../logs

firefox opening.

At index.hux
  Taking screenshot firefox-1.png
[1]+  Done                    selenium

Ben@BEN-PC ~/Desktop/blarg
$
bzuillsmith commented 10 years ago

This may or may not be important. I tried it with chrome and got the same behavior, but the chrome driver produced some logs. Note the pipe error only happens about the time it is failing:

[8912:7424:0312/110853:ERROR:chrome_views_delegate.cc(185)] NOT IMPLEMENTED
[8912:7424:0312/110853:ERROR:desktop_root_window_host_win.cc(760)] NOT IMPLEMENT
ED
[8912:7424:0312/110853:ERROR:desktop_root_window_host_win.cc(760)] NOT IMPLEMENT
ED

... repeats a few times, then ...

[8696:7472:0312/110908:ERROR:ipc_channel_win.cc(132)] pipe error: 109
[5736:6484:0312/110908:ERROR:ipc_channel_win.cc(405)] pipe error: 232
[8532:9452:0312/110922:ERROR:extension_icon_image.cc(201)] Start loading extensi
on icon for Chrome. scale = 1
[8532:9452:0312/110922:ERROR:extension_icon_image.cc(201)] Start loading extensi
on icon for Store. scale = 1
[8532:9452:0312/110922:ERROR:desktop_root_window_host_win.cc(760)] NOT IMPLEMENT
ED
[8532:9452:0312/110922:ERROR:extension_icon_image.cc(219)] Component extension i
con for Chrome is loaded. scale = 1
[8532:9452:0312/110922:ERROR:extension_icon_image.cc(219)] Component extension i
con for Store is loaded. scale = 1
[8912:7424:0312/110923:ERROR:extension_icon_image.cc(201)] Start loading extensi
on icon for Chrome. scale = 1
[8912:7424:0312/110923:ERROR:extension_icon_image.cc(201)] Start loading extensi
on icon for Store. scale = 1
[8912:7424:0312/110923:ERROR:desktop_root_window_host_win.cc(760)] NOT IMPLEMENT
ED
[8912:7424:0312/110923:ERROR:extension_icon_image.cc(219)] Component extension i
con for Chrome is loaded. scale = 1
[8912:7424:0312/110923:ERROR:extension_icon_image.cc(219)] Component extension i
con for Store is loaded. scale = 1
chenglou commented 10 years ago

Edit: nvm. Saw it was just a screenshot. Will check this.

chenglou commented 10 years ago

Also, I've encountered weird errors with outdated Selenium and Chrome drivers. Can you update to see if this is resolved?

bzuillsmith commented 10 years ago

I've got Chrome Driver v2.9 for win32 and Selenium 2.38.0 I wouldn't think it's a Chrome Driver issue since I get the same behavior from Firefox

chenglou commented 10 years ago

I can't repro this. I guess this might be a windows problem. The update (automatically done right after recording) and playback share mostly the same code, aside from overriding screenshot rather than comparing them and Selenium has nothing to do with this. If you do hux -u rather than hux at the last step, does that change anything?

bzuillsmith commented 10 years ago

Hmmm -u causes it to hang with no output. I'll run it in the node-inspector and see if there is anything obvious I can find Edit: actually, selenium & hux -u starts selenium and then no output from hux, and it hangs. But starting selenium separately and calling hux -u worked. hux still has the same issue as before however.

chenglou commented 10 years ago

Geez, thanks a lot for spending the whole debugging session today! Sorry again for the dumb mistake that the tests somehow didn't cover. Fixed and released as 0.7.2. Keep on giving feedbacks please!