streaming >4GB-files doesn't come to an end.

[felixniemeyer]

I'm trying to stream a compressed torrent file and pipe it to a decompressor. node stream.js | bzip2 -dkc

It works fine for some files but fails for some other files - conspicuously it fails for files with sizes > 4GB: The failure is, that the readStream never finishes. The readStream never emits end and the file in /tmp/torrent-stream/... remains incomplete. Only very little data is missing at the end of the file. No error get's emitted by the readStream.

You can run this script to reproduce (the torrent is about reddit comments):

var torrentStream = require('torrent-stream');
var fs = require('fs')

var magnetLink = process.argv[2] || 'magnet:?xt=urn:btih:7690f71ea949b868080401c749e878f98de34d3d&dn=reddit%5Fdata&tr=http%3A%2F%2Ftracker.pushshift.io%3A6969%2Fannounce&tr=udp%3A%2F%2Ftracker.openbittorrent.com%3A80'
var fileName = process.argv[3] || 'RC_2014-01.bz2'

var engine = torrentStream(magnetLink);

engine.on('ready', () => {
  var hasMatch = false;
    engine.files.forEach(file => {
    if(file.name == fileName) {
      hasMatch = true; 
      stream = file.createReadStream()      
      stream.pipe(process.stdout)
      stream.on('end', () => {
        engine.remove(false, () => {
          engine.destroy()
        }) 
      })
    }
    });
  if(!hasMatch) engine.destroy()
});

I tried to download the problematic file with a torrent client and it worked.