busterjs / buster

Abandoned - A powerful suite of automated test tools for JavaScript.
http://docs.busterjs.org
Other
448 stars 37 forks source link

buster-ci should not allow additional browsers to be attached at the end of the process. #449

Open GCheung55 opened 9 years ago

GCheung55 commented 9 years ago

Somehow, additional browsers are being attached at the end of the test runs. Browsers should only attach at the beginning and prevented when close to the end of the process.

     [exec] validateBrowserConfig
     [exec] capture browsers
     [exec] ["1"]
     [exec] sendMessage: {
     [exec]   browsers: { Phantom: { id: 1 } },
     [exec]   command: "start",
     [exec]   url: "http://localhost:1313/capture"
     [exec] }
     [exec] received command: {
     [exec]   browsers: { Phantom: { id: 1 } },
     [exec]   command: "start",
     [exec]   url: "http://localhost:1313/capture"
     [exec] }
     [exec] start browser Phantom
     [exec] slave ready: { slaveId: "1" }
     [exec] []
     [exec] All browsers are ready.
     [exec] run tests
     [exec] slave ready: { slaveId: "fbf293ab-c4ca-4133-80ab-58a5a12f6c55" }
     [exec] []
     [exec] All browsers are ready.
     [exec] close browsers
     [exec] ["1"]
     [exec] slave disconnected gracefully: { slaveId: "b8ddbe66-fac6-42ed-8255-c17a5c9ae959" }
     [exec] []
     [exec] All browsers are closed.
     [exec] All done.
     [exec] sendMessage: { browsers: { Phantom: { id: 1 } }, command: "stop" }
GCheung55 commented 9 years ago

I'm beginning to believe that the additional slave browser are actually the same slave browser. What I think is happening is that when the tests are run, a synchronous execution is blocking the server from responding to the slave browsers. So the slave browsers think there's a disconnect and will attempt to reconnect. Once the synchronous execution is done, the server can respond to the slave browsers and reconnect. When the reconnect occurs, the page technically refreshes and is given a new slaveId.

GCheung55 commented 9 years ago

Here's another log.

     [exec] validateBrowserConfig
     [exec] capture browsers
     [exec] ["1"]
     [exec] sendMessage: {
     [exec]   browsers: { Phantom: { id: 1 } },
     [exec]   command: "start",
     [exec]   url: "http://localhost:1111/capture"
     [exec] }
     [exec] received command: {
     [exec]   browsers: { Phantom: { id: 1 } },
     [exec]   command: "start",
     [exec]   url: "http://localhost:1111/capture"
     [exec] }
     [exec] start browser Phantom
     [exec] Test-JS PhantomJS PID: 37291
     [exec] 
     [exec] slave ready: { slaveId: "1" }
     [exec] []
     [exec] All browsers are ready.
     [exec] run tests
     [exec] slave ready: { slaveId: "74d6ab3f-7e71-4e3e-9879-07fd3769223a" }
     [exec] []
     [exec] All browsers are ready.
     [exec] close browsers
     [exec] ["1"]
     [exec] All done.Not all browsers could be closed!: 1
     [exec] 
     [exec] sendMessage: { browsers: { Phantom: { id: 1 } }, command: "stop" }
GCheung55 commented 9 years ago

Perhaps buster-server-cli should run in a child process fork?

GCheung55 commented 9 years ago

I've also found that a REALLY long process could render the agent inaccessible because I'm seeing the following message.

Agent http://localhost:18888 not accessible!

I'm not sure if the browser is going idle would cause Faye Client to report that.

GCheung55 commented 9 years ago

Perhaps separating the buster-test-cli to run in a separate child process would also be a good idea.

dwittner commented 9 years ago

I am not convinced yet, that the cause of your problems has something to do with blocking synchronous tasks. First of all i wonder why a browser gets a new session id after reconnecting. I will clarify that first and then investigate your problem further.

dwittner commented 9 years ago

@GCheung55, i have just released version 0.2.2 of buster-ci, which now captures a phantom browser, if you add configuration option server.runPhantom: true, see https://github.com/dwittner/jquery-clipboard/blob/master/buster-ci.js for an example. Maybe that helps you to solve this issue.

dwittner commented 9 years ago

@GCheung55, is your issue solved?

GCheung55 commented 9 years ago

I will check in the morning.

-Garrick-

On Jul 13, 2015, at 8:12 AM, Daniel Wittner notifications@github.com wrote:

@GCheung55, is your issue solved?

― Reply to this email directly or view it on GitHub.

GCheung55 commented 9 years ago

Didn't seem to address the issue.

Here's the logged output.

Capture browsers
["0"]
Starting headless browser...
Headless browser was captured.
Slave ready: { slaveId: "0" }
[]
All browsers are ready.
Run tests
Slave ready: { slaveId: "31c95d89-7cc4-4ca2-8024-540fc7e0d418" }
[]
All browsers are ready.
Close browsers
No browsers have to be closed.
All done.

As you can see, there are two slave ID's.

The buster-ci config:

module.exports = {
    outputFile: outputPath,

    server: {
        host: serverHost,
        port: serverPort
        runPhantom: true
    },

    // browsers: {
    //     Phantom: {
    //         start: phantomjsPath,
    //         startArgs: [
    //             'pjs-runner.js'
    //         ]
    //     }
    // },

    // agents: {
    //     localhost: {
    //         port: 18888,
    //         browsers: ['Phantom']
    //     }
    // },

    captureTimeout: 40,
    closeTimeout: 40,

    // logLevel: 'warn'
    logLevel: 'debug'
};

The pjs-runner.js file is the script to have a phantomjs instance open the url to get captured. You can disregard that.

GCheung55 commented 9 years ago

I am not convinced yet, that the cause of your problems has something to do with blocking synchronous tasks. First of all i wonder why a browser gets a new session id after reconnecting. I will clarify that first and then investigate your problem further.

I think this can be easily reproduced with buster-server with the following steps:

  1. Spin up buster-server and capture a browser.
  2. Once the browser is captured, take note of the slave ID in the URL and kill buster-server.
  3. Spin up buster-server again and take a look at the captured browser's URL and compare the slave ID.

What happens here is that once the buster-server is killed the first time, the captured browser will attempt to reconnect with the server via faye. Once the buster-server is available again, a new slave ID is assigned and the page is refreshed. @augustl might have some insight in this area?

GCheung55 commented 9 years ago

I forgot to mention how block synchronous tasks is related to the killed-buster-server example I described above.

My understanding is that slave browser will periodically ping the buster-server to make sure a connection is still available. But since the browsers, buster-server, and buster-test all run in one instance, the event loop is taken up and can not respond to the ping, making the browsers think that the buster-server is down thus causing attempts to reestablish a connection.