backpressure not handled

[SebastianTusk]

I'm importing data with the following code snippet.

const loadDataQuery = `
    LOAD DATA LOCAL INFILE 'stream' INTO TABLE import_table FIELDS TERMINATED BY '|' LINES STARTING BY '0' TERMINATED BY '\n'
`;
const [loadDataResults] = await connection.query({
    sql: loadDataQuery,
    infileStreamFactory: (path: string) => {
        return loadDataStream;
    }
});

If the data stream produces data very fast the mysql connection's socket has to buffer all this data. This leads to an out of memory error.

It looks like there is code that tries to handle this condition at https://github.com/sidorares/node-mysql2/blob/dbf48879f517b8ebefc9fbdc5508a20b84f833ec/lib/commands/query.js#L136 The problem is that the socket never issues any pause events. Backpressure has to be handled by using the return value of writable.write.

[sidorares]

yeah, this is old code in the times of Streams v1, we need to modernise it to a newer streams api Would be good to have your help here, looks like you are in good position to debug