Can't get the whole file sometimes

[taozi0818]

I want to download some files from FTP server.But i cannot get the whole file when i use get() method. For example, the size of file which i want to download is 200000 bytes, when i get 199000 bytes (99%) of the file, the download progress will pause or stop.But, stream and connection can't catch any errors or trigger any events(trigger close event sometimes).

[icetee]

Yes, sometimes it happens with me. I don't know why it does.

[taozi0818]

I resolved this with another library.... But something go wrong when connect to FTP server with that library.

[Bestulo]

Which library did you use?

[taozi0818]

@Besatnias jsftp. But this library is not good at handling error.you cannot detail information about error sometimes.

[JourdanClark]

I'm getting this issue as well. I'm needing to download about 30gb split into 3gb files every day. When the script is running on gigabit internet, all the downloads succeed. When running on 50mbps internet, the downloads will just completely stop if they don't download fast enough. I don't think it's the ftp server that's dropping me since I can download them through chrome just fine using the ftp://url

[JourdanClark]

I'm also listening to every event on stream and client and nothing happens. I also noticed that on slower internet, when I was downloading a 110mb file, the download would finish and it would just sit there for about 40 seconds before it would close the stream and end the client. On fast internet, the client would end as soon as the download finished.

[taozi0818]

@JourdanClark You can test it again with this library jsftphttps://github.com/sergi/jsftp. Hope it can help you.

[rlueder]

For those also having issues with not getting the whole file, it seems related to small files over fast networks as described by @polidore on https://github.com/mscdex/node-ftp/issues/70#issuecomment-34785017 (the file I was having issues with was 142kb over a 700Mbps connection). This snippet worked for me (based on a workaround by @ugate on https://github.com/mscdex/node-ftp/issues/66):

const FTP = new ftp();
const FILE = "MyFile.txt";
const PASSTHROUGH = new require("stream").PassThrough();

FTP.get(FILE, (error, stream) => {
  if (error) {
    console.log(`ERROR: ${error.message}`);
    throw error;
  } else {
    stream.pipe(PASSTHROUGH),
      FILE,
      error => {
        if (error) {
          console.log(`ERROR: ${error.message}`);
          throw error;
        } else {
          // in my case I'm writing to a temporary file, but you can do whatever is needed at this point
          fs.createWriteStream(path.join("/tmp", "data.tsv"));
        }
      };

    PASSTHROUGH.end(() => {
      FTP.end();
      console.log("Closed FTP connection. Goodbye!");
    });
  }
});

Documentation on stream.PassThrough can be found here: https://nodejs.org/api/stream.html#stream_class_stream_passthrough