pm2 cluster duplicate data inserted in database whene reading stream

[syahiaoui]

Your issue may already be reported! Please search on the issue tracker before creating one.

What's going wrong?

Duplicate data inserted to database

How could we reproduce this issue?

//file.js
const processFile = () => {
    const lineReader = readline.createInterface({
        input: fs.createReadStream(initFile, {
            highWaterMark: 256
        })
    });
    lineReader.on('line', async (line)=> {
        let result;
        try {
            result = await bucket.listAppend(my_key, values, options);
        } catch (error) {
            console.error(error);
        }
    });
    lineReader.on('error', (error) => {
        console.error(error);
    });
    lineReader.on('close', () => {
        console.log("END");
    });
};

Read huge file (2Go) using stream line by line, after that insert this line to couchbase database. I use: pm2 start file.js -i max to start my application

Supporting information

Please run the following command (available on PM2 >= 2.6)

$ pm2 report
-- PM2 REPORT (Thu Mar 22 2018 17:15:29 GMT+0100 (Paris, Madrid)) ------------
===============================================================================
--- Daemon -------------------------------------------------
pm2d version         : 2.7.2
node version         : 9.8.0
node path            : C:/Program Files/nodejs/node
argv                 : C:\Program Files\nodejs\node.exe,C:\Users\AppData\Roaming\npm\node_modules\pm2\lib\Daemon.js
argv0                : node
user                 : undefined
uid                  : N/A
gid                  : N/A
uptime               : 3039min
===============================================================================
--- CLI ----------------------------------------------------
local pm2            : 2.7.2
node version         : 9.8.0
node path            : C:/Program Files/nodejs/node
argv                 : C:\Program Files\nodejs\node.exe,C:\Users\AppData\Roaming\npm\node_modules\pm2\bin\pm2,report
argv0                : C:\Program Files\nodejs\node.exe
user                 : undefined

[wallet77]

Hi @syahiaoui

I really don't understand your problem. PM2 can't duplicate data. If data is duplicated then many instances have been called to do the work.

But PM2 can't do this kind of "magic".