Bugfix/500 errors

[corycaywood]

Resolves #13

Added incremental retries - each retry will take twice as long as the previous retry.

Added a queue using the "promise-request" module so that no more than 29 connections can be made at once. This has reduced the number of request retries caused by 500 errors.

[coveralls]

Coverage increased (+0.004%) to 98.802% when pulling 64542ebfec6d53b53de62120eeed5d444c3a33ee on bugfix/500-errors into 92310973d0b707531e10498bb0e055b40c6e9de4 on master.

[SwinX]

Added a queue using the "promise-request" module so that no more than 29 connections can be made at once. This has reduced the number of request retries caused by 500 errors.

Was it measured somehow? How much articles and sections were in category you uploaded? Were there any cases when uploading failed with error?

[coveralls]

Coverage increased (+0.01%) to 98.81% when pulling 882c4efa4a2f2c57662497c90b57359a094f36c5 on bugfix/500-errors into 92310973d0b707531e10498bb0e055b40c6e9de4 on master.

[corycaywood]

The issue with retries due to 500 errors occurs most often during resource uploads because they maintain a connection for a longer time period. So I tested on a repo with 80 resources.

Before implementing this solution, the tool would log about 30 retries in a row. Now it logs 10 or less.

I tested how many connections can be made at once before Zendesk returns a 500 error, and it was around 30 connections, so that's why I chose 29 as MAX_CONCURRENT, but this number maybe still needs to be adjusted.

[coveralls]

Coverage increased (+0.01%) to 98.81% when pulling 397e95c8d4ba096a060e9ecf393297d457ae3d88 on bugfix/500-errors into 92310973d0b707531e10498bb0e055b40c6e9de4 on master.

[coveralls]

Coverage increased (+0.01%) to 98.81% when pulling 9a09dfa5602a9ef5f6f0c2121cac60744512461c on bugfix/500-errors into dc07ee27c08e2f5e9d5c7b17013b88af08054e49 on master.