segment-boneyard / nightmare

A high-level browser automation library.
https://open.segment.com
19.55k stars 1.08k forks source link

nightmarejs , hangs the browser #985

Closed 1gluk closed 7 years ago

1gluk commented 7 years ago

Hello. Please help. I do not understand what the problem is a nightmare. The essence of the application logic is as follows: in the database has a list of tasks for a nightmare, when measuring state is generated by the initialization process of a nightmare. The problem sludyuschem, at some point in time copies of a nightmare to stop responding and hang. On the counters in the database it can be seen that they can work for longer than specified in the timeout settings nightmare. Please tell me what could be the problem and how it can be combated. Check the status and run:

const processParse = require('./processParse');
//...
r.table('arc_tasks').getAll('toParse', {index: 'status'}).changes().run(conn, (err, cursor) => {
            if (err) throw err;
            cursor.each((err, data) => {
                if (err) throw err;
                let val = data.new_val;
                if (val !== null) {
                    val.start = Date.now();
                    processParse(val).catch(console.error); // <- this is run process nightmare
                }
            });
        });
// ...

An exemplary process for the code of a nightmare: processParse:

// requires and etc
module.exports = exports = async function processParse(params){
    let nightmare;
    try{
        await nighmare.authentication(params.user, params.pass);
        await nightmare.goto(params.href) // .click....click....;
        let resp = await nightmare.evaluate((selector)=>{
            return new Promise((resolve,reject)=>{
                setTimeout(reject, 30000, new Error('EVAL_TIMEOUT'))
                //....
                resolve(result);
                // ... 
                reject(result);
                });
            }, params.selector);
        await nightmare.end();
    } catch(error){
        await nightmare.end();
        throw new Error('PARSER_ERROR');
    } finally {
        // save results and change state of task
    }
};

Example statistics (RUNTIME IN SECS):


        "href": "https://***",
        "templName": "GetModifications",
        "runtime": 505.369
    },
    {
        "href": "https://***",
        "templName": "GetModifications",
        "runtime": 1126.206
    },
    {
        "href": "https://***",
        "templName": "GetModifications",
        "runtime": 860.733
    },
    {
        "href": "https://***",
        "templName": "GetModifications",
        "runtime": 379.539
    }```
1gluk commented 7 years ago

use a temporary solution, it is not like this.

nightmareHaltKey = setTimeout((nightmare) => {
                nightmare.halt(new Error('NIGHTMARE_FORCE_KILL'));
            }, 120000, nightmare);
rosshinkley commented 7 years ago

How are your Nightmare instances getting created? It looks like the instance isn't ever created based on your source. Also, what version are you using? Are you getting errors for the instances that are hanging?

1gluk commented 7 years ago

Hi, Rosshinkley. For each job created another instance through let nightmare = Nightmare(config). Every nightmare is an instance of a large list of tasks, including the creation of screenshots, download files. I guess there are jobs that can potentially lock code on the browser side, after which the connection between the processes is terminated. I used .halt method, but it does not always help. I came to the idea to explicitly use fkill electron off (nightmare.proc.pid). Separately, I note that the assignments that I give not block browser evalyuate. In addition, I note that there is no electron closing event, if the electron close the arms, the nightmare did not know about it. But this is the problem of the above does not apply, a further observation.

1gluk commented 7 years ago

parsing error in the example given probation, I react to all exceptions nightmare if they occur. In my situation, the process is sometimes just stops without reference to a fault block.

casesandberg commented 7 years ago

Are you still having trouble and have you tried 2.10.0? I am going to close this, but feel free to re-open if it's still an issue.