nodejs / node

Node.js JavaScript runtime ✨🐢🚀✨
https://nodejs.org
Other
107.69k stars 29.64k forks source link

nodejs download some of file fine but some empty with errno: 'ETIMEDOUT' #4556

Closed pooyamoha closed 8 years ago

pooyamoha commented 8 years ago

I have tried to download 1187 image url (stored in log text file) and download them in my local machine , most of the file are downloaded but i get TimeOut Error for some of them. (the total image file might be around 866 file, because some of the url are duplicate. i would appreciate if someone with modifying my script source code show me , how can i find a fix or workaround for the timeout issue and get download all the files fine. i uploaded the stored text file and the script here, so you can test it.

click here to download stored url log.txt

if you want to test my situation , you need to download above attached file and put it in the same folder of script file and run the script.

here is the source code :

 var fs = require('fs'),
        request = require('request');
    var path = require('path');

var lineReader = require('readline').createInterface({
  input: require('fs').createReadStream('log.txt')
});

lineReader.on('line', function (line) {
  console.log('Line from file:', line);
var img_fileName = path.basename(line);

 var file = fs.createWriteStream(img_fileName,{ flags: 'w'});

var options = {
    url: line,
    port: 80,
    setTimeout: 0,
    setNoDelay: true,
    setSocketKeepAlive: true    
};

request(options, function(err,response,body) {

 if (err)
 console.log(err);

 }).pipe(file);;

});

i get this error in console log on some of the files but most of them download fine and those file that get timeout when download them , store as a empty file.

{ [Error: connect ETIMEDOUT 192.82.104.198:80]
  code: 'ETIMEDOUT',
  errno: 'ETIMEDOUT',
  syscall: 'connect',
  address: '192.82.104.198',
  port: 80 }

P.S :

npm -v : 3.5.3 ######### node -v : 5.3.0

bnoordhuis commented 8 years ago

Can you post this here or on SO? This repo is for reporting bugs against node.js core, not general support. Thanks.