Duplicate or Invalid values exists in buffer when writing file

[aceben81]

I coded it as shown in the example, and I was very satisfied. Save the file in the middle when sending another server based on http-proxy-middleware.

There is no problem when receiving files as streams at once. However, there are duplicate and invalid content when receiving and saving files when cutting and sending files from a client multiple times.

There seems to be a problem when writing again after writing all the data that came in.

const { createProxyMiddleware } = require('http-proxy-middleware');
app.use('/api/file/**', createProxyMiddleware({
        target: TARGET_URL,
        logLevel: 'debug',
        changeOrigin: true,
        onProxyReq: restream
}));

Test case 1

var restream = function(proxyReq, req, res, options) {
const bb = busboy({ headers: req.headers });
bb.on('file', (name, file, info) => {

            const { filename, encoding, mimeType } = info;
            const uploadPath = path.resolve(__dirname, '../uploads', filename);

            console.log(
                    `File [${name}]: filename: %j, encoding: %j, mimeType: %j`, filename, encoding, mimeType
            );

            file.pipe(fs.createWriteStream(uploadPath, {flags:'a'}));
    });

    bb.on('close', () => {
            console.log('Done parsing form!');
    });

    req.pipe(bb);

}

> Test case 2

var restream = function(proxyReq, req, res, options) { const bb = busboy({ headers: req.headers }); bb.on('file', (name, file, info) => {

            const { filename, encoding, mimeType } = info;
            const uploadPath = path.resolve(__dirname, '../uploads', filename);

            console.log(
                    `File [${name}]: filename: %j, encoding: %j, mimeType: %j`, filename, encoding, mimeType
            );
            const writeStream = fs.createWriteStream(uploadPath, {flags:'a'});
            file.on('data', (data) => {
                    console.log(`File [${name}] got ${data.length} bytes`);
                    writeStream.write(data);
            }).on('close', () => {
                    console.log(`File [${name}] done`);
                    writeStream.end();
            });
            bb.on('close', () => {
            console.log('Done parsing form!');
    });

    req.pipe(bb);

}

![Screen Shot 2022-09-22 at 8 51 26 PM](https://user-images.githubusercontent.com/114155267/191741326-9bdd9b11-4a70-4fee-9ce9-d4e54fdea8f6.png)

Original buffer
Written bffer
![Screen Shot 2022-09-22 at 8 53 59 PM](https://user-images.githubusercontent.com/114155267/191741438-7bbb15d3-93b2-4585-b5be-c689ea077dac.png)
![Screen Shot 2022-09-22 at 9 02 47 PM](https://user-images.githubusercontent.com/114155267/191741563-adb4251c-1ac2-4407-8d08-9e18e1dacc06.png)

[mscdex]

I don't see anything wrong with the two test cases you've provided, except in the second case you're not using the stream backpressure mechanism (e.g. when stream.write() returns false and starts buffering more data in memory).

Have you tried removing third party modules (e.g. http-proxy-middleware) to ensure they are not causing the problem you're having?

[aceben81]

I didn't test except http-proxy-middleware. I will try to test only using express and busboy. Thank you for your answer.