Automattic / knox

S3 Lib
MIT License
1.74k stars 285 forks source link

ECONNRESET on putFile #198

Closed zthomas closed 11 years ago

zthomas commented 11 years ago

I'm keep getting ECONNRESET from knox even though I'm attaching an error handler on the returned request object. This causes my server to crash every time and it's really frustrating. It's also strange that statuscode != 200 doesn't err out but that's another story. I also use putStream in the app frequently and that doesn't err out at all.

I think it's an error with knox, we should be able to have a clean way to handle any errors. Anyone know what's going on here?

Environment

node: 0.10.5 knox: 0.8.4 express: 3.3.3 mode: production

My putFile Call

var  S3 = Knox.createClient({
  key:  Conf.awsKey,
  secret: Conf.awsSecret,
  bucket: Conf.awsBucket
}),
S3.putFile(fileInfo.relPath, fileInfo.s3Path(), Conf.awsPublic, function(err, resp){
    if (err || (resp && resp.statusCode != 200)) {
      logger.error(err, '\nFileInfo' + fileInfo, fileInfo.s3Path());
      // some fallback logic...
      return cb(null, true);
    } else {
      fileInfo.url = fileInfo.s3Url();
      return cb();
    }
  }).on('error', My.logErr);

My trace with LongJohn:

Error: read ECONNRESET at errnoException (net.js:884:11)

at onread (net.js:539:19)

at Readable.on (_stream_readable.js:663:33)
at http.js:1705:12
at process._tickCallback (node.js:415:13)

at ClientRequest.onSocket (http.js:1682:11)
at new ClientRequest (http.js:1392:10)
at exports.request (https.js:123:10)
at Client.request (/var/www/engage/node_modules/knox/lib/client.js:306:42)
at Client.put (/var/www/engage/node_modules/knox/lib/client.js:347:15)
at Client.putStream (/var/www/engage/node_modules/knox/lib/client.js:428:18)
at /var/www/engage/node_modules/knox/lib/client.js:398:20
at Object.oncomplete (fs.js:107:15)
domenic commented 11 years ago

You're not doing anything with the response object. You should pipe it somewhere, or call resp.resume(), and probably should attach an error handler to it.

zthomas commented 11 years ago

Ahh thanks for the quick reply. I just saw the issue from a few days ago https://github.com/LearnBoost/knox/issues/192

That is pretty annoying. Is there some way for knox to handle that for us instead and wrap that response (and maybe check the statusCode as well). A lot of new users won't be familiar with having to do something extra with res from a callback and will probably run in to the same exact issue that we had.

domenic commented 11 years ago

Yeah, that's planned, #114, but nobody's had the bandwidth to implement it yet. Sorry about that :(.

samholmes commented 10 years ago

I'm having the same error. How should one properly handle the res object? I need an example of how to do it.

I added res.resume() to all the callbacks of the methods, and I'm still getting these errors:

Error: spawn ENOENT
  at errnoException (child_process.js:988:11)
  at Process.ChildProcess._handle.onexit (child_process.js:779:34)

Error: write EPIPE
  at errnoException (net.js:904:11)
  at Object.afterWrite (net.js:720:19)

I also get the ECONNRESET error, but they all are the same ambiguous shit. I can't seem to solve this issue, and it's been all day.

samholmes commented 10 years ago

Turns out my issue was that I forgot to install GraphicsMagick for the gm module. It had nothing to do with knox. -.-

However, I'd still like to know what to do with the res object and how to properly handle it. Is res.resume() all that's needed, or do I need to listen for error events on it?

Thanks.

domenic commented 10 years ago

You should definitely listen for error events on the response object (and in general on all streams). Otherwise, if they occur, your program will crash.

dineshviswanath commented 10 years ago

Following code helped to fix this read ECONNRESET error. Thanks for saving my day.

 var s3req = client.put(...);

s3req.on('error', function(err){
 /* This error handler fixs the following error. .
 read ECONNRESET */
  sails.log.info("knox Amazon S3 request Connection error due to "+err);
  });