search

gruntjs / grunt-lib-phantomjs

Grunt and PhantomJS, sitting in a tree.
http://gruntjs.com/
MIT License
93 stars 63 forks source link

Use WebSockets for IPC instead of file I/O #36

Open JamesMGreene opened 11 years ago

JamesMGreene commented 11 years ago

Using grunt-contrib-qunit (and likely the similar plugin tasks for other test frameworks) is incredibly slow on Windows due to the very chatty "eventing" file I/O that the two libraries exchange. Since Windows sucks at file I/O (file open-lock-write-close), this makes using grunt-contrib-qunit on our fairly large existing test base take ~22 minutes versus the ~4 minutes it takes when we run them via our own non-chatty PhantomJS script.

To prove that the file I/O was the primary cause of the slowness, we also tried switching over to using streaming stdout/stderr instead of file I/O. This cut the time down to ~9 minutes but that was still much too slow compared to the current ~4 minutes.

I would love to see grunt-contrib-qunit/grunt-lib-phantomjs use WebSockets for IPC with PhantomJS instead of file I/O. The only major caveat is that the current versions of PhantomJS (1.x) only support hixie-76 draft WebSockets, so it can be a little tricky in getting them to work with other WebSockets libraries that are geared toward the modern specs. However, I know it has been successfully done by the Poltergeist project, using Ruby's faye-websocket and later websocket-driver), the phantom Node module using SockJS, and likely others.

Perhaps try out the Faye project's websocket-driver module for Node or SockJS?

jsoverson commented 11 years ago

Agreed that the file io communication is not optimal, there's also issue https://github.com/gruntjs/grunt-lib-phantomjs/issues/6 that might be a better implementation route.

A PR taking care of this would be greatly appreciated. It's recognized as an issue but I don't have time to address it, especially since I never see the pain of implementations on windows.

JamesMGreene commented 11 years ago

@jsoverson: Issue #6 only changes the communication mechanism between PhantomJS and its WebPage instances, not between Node and PhantomJS.

I actually tried to fix this issue during some hallway hack time at the jQuery Conference in Portland this year but my unfamiliarity with WebSockets seemed to foil me repeatedly. I was, however, also using other Node libraries than the ones I mentioned here, none of which seemed to pan out; namely: socket.io, websocket.io, and faye-websocket-node.

Arkni commented 8 years ago

@JamesMGreene

I succeed in using WebSockets for communication using ws. See #102 If you have the time, please give it a try and provide some feedback :)