search

rakutentech / docpub

DocPub is a command line utility for converting a folder structure of markdown files to HTML and uploading the result to Zendesk.
MIT License
12 stars 16 forks source link

Socket Hang Up error #13

Closed corycaywood closed 7 years ago

corycaywood commented 7 years ago

Need to investigate what causes this error:

Upload failed!
socket hang up
Error: socket hang up
   at createHangUpError (_http_client.js:302:15)
   at TLSSocket.socketOnEnd (_http_client.js:394:23)
   at emitNone (events.js:91:20)
   at TLSSocket.emit (events.js:186:7)
   at endReadableNT (_stream_readable.js:974:12)
   at _combinedTickCallback (internal/process/next_tick.js:74:11)
   at process._tickCallback (internal/process/next_tick.js:98:9)
corycaywood commented 7 years ago

I haven't been able to reproduce the socket hang up error. However when running the tool on a repository with a large number of files, ZendeskClientWrapper will often need to retry uploads because it gets a 500 error from Zendesk.

The 500 errors seem to happen most often on the resource uploading step, when many connections are staying open at once. So I think it happens because of a limit by Zendesk's servers regarding the number of simultaneous connections. After some testing, I found that 500 errors seem to happen when more than 30 connections are made at once.

So I propose two things for this issue:

  1. Implement request throttling so that no more than 30 connections are made at once.
  2. Implement incremental "Retry After" times - each retry should wait an incrementally longer amount of time before retrying again:
    • 1st retry: 500ms
    • 2nd retry: 1000ms
    • 3rd retry: 2000ms
    • 4th retry: 4000ms
    • 5th retry: 8000ms