TritonDataCenter / node-manta

Node.js SDK for Manta
75 stars 54 forks source link

Proper way to abort a request? Undocumented option no_resume? #331

Open doublerebel opened 7 years ago

doublerebel commented 7 years ago

Hello,

I have a situation where I'm syncing with manta directly to/from from http streams. The main issue is, there doesn't seem to be an official way to abort the GET request. I've traced up and down the stack, and the best way seems to be to call res.destroy().

Why I need it:

  1. I first check to see if the file exists in Manta (and attach an error handler to the stream from Manta). I need the file metadata, so I can't just .ls() the directory.
  2. Then I check to see if I have a matching file locally
  3. If I do not have a matching file, I stream the file from Manta to local
  4. If I do have a matching file, I abort the request (right now using res.destroy())

Where the problem is:

If I don't abort the request, I eventually receive this error to my error handler, I assume because I left an open connection:

{ Error: read ECONNRESET
    at exports._errnoException (util.js:1024:11)
    at TCP.onread (net.js:610:25) code: 'ECONNRESET', errno: 'ECONNRESET', syscall: 'read' }

but, there appears to be retry logic where the original res is left behind and a new download is started, and I have no way to access the new res. This is where no_resume comes in: https://github.com/joyent/node-manta/blob/master/lib/client.js#L870

When the connection is interrupted and the download auto-retries, the new res never is passed to a callback because the callback is restricted by once: https://github.com/joyent/node-manta/blob/master/lib/client.js#L818

Therefore I can't cancel the new res because I have no access to it. If I pause() the stream it eventually resets the connection, if I destroy() or end() the stream I get a write after end error.

I've searched through the codebase but I don't see another way. no_resume doesn't really seem to be a solution because I need to download the file despite a dodgy connection. Is there something I'm missing? My initial thought is to just add a cancel/abort method to the stream object that is passed to the callback, which takes action on the current res.

EDIT: I did just discover minfo, which has doc notes in the .js file but is not documented in the Node SDK docs. Useful but doesn't exactly solve my problem.

EDIT 2: I forgot to mention, I also looked through all through restify-client and find no specific way to abort requests in restify-client.

davepacheco commented 7 years ago

Yes, I think the root problem here is that although Node's http client library provides an abort() method, neither the underlying restify client nor the Manta client actually returns a request object, so there's no handle on which to call abort (and certainly no path to invoking the built-in request.abort()).

Have you considered using the .info() method to make a HEAD request instead? Then you would make a GET request only if needed. It's an oversight the .info() method isn't documented -- I believe it's reasonably widely used.