READY: Improve closed connection handling

[lukebakken]

Fixes #137 (CLIENTS-785)

Add unique ID to each command's name to facilitate debugging. Make re-try happen after a 125ms interval. Health check is now on 75ms interval. Fix bug in RiakConnection where closed sockets during Health Check would not raise correct event.

[lukebakken]

@mogadanez - if you have time, can you try this branch in your environment? I am working on handling unexpected socket closures better. Thanks!

[mogadanez]

did not work directly from git dependency:

fs.js:584
  return binding.open(pathModule._makeLong(path), stringToFlags(flags), mode);
                 ^

Error: ENOENT: no such file or directory, open '/var/proofme/node_modules/basho-riak-client/lib/protobuf/riak_pb/src/riak_pb_messages.csv'
    at Error (native)
    at Object.fs.openSync (fs.js:584:18)
    at Object.fs.readFileSync (fs.js:431:33)
    at Object.<anonymous> (/var/proofme/node_modules/basho-riak-client/lib/protobuf/riakprotobuf.js:19:4)
    at Module._compile (module.js:413:34)
    at Object.Module._extensions..js (module.js:422:10)
    at Module.load (module.js:357:32)
    at Function.Module._load (module.js:314:12)
    at Module.require (module.js:367:17)
    at require (internal/module.js:16:19)
    at Object.<anonymous> (/var/proofme/node_modules/basho-riak-client/lib/core/riakconnection.js:11:20)

can you publish it under beta tag in npm? https://gist.github.com/AvnerCohen/4051934#publish-a-package-not-under-the-default-latest-tag

[lukebakken]

@mogadanez - this project uses git submodules, which npm won't correctly clone.

I use npm link and this method for local testing of the module. This should work for you, be sure to clone riak-nodejs-client with the --recursive argument.

I'm in the process of publishing a pre-release version at the moment, as well.

[lukebakken]

@mogadanez - I have modified build/publish in this branch and it appears that 2.1.2-beta2 should be ready for you to use:

lbakken@brahms ~
$ npm dist-tag ls basho-riak-client
beta: 2.1.2-beta2
latest: 2.1.1

Let me know how it works. Thank you for providing information about non-latest npm packages.

[lukebakken]

beta3 is now available for testing:

$ npm dist-tag ls basho-riak-client
beta: 2.1.2-beta3
latest: 2.1.1

[lukebakken]

@mogadanez - any chance to test beta3?

[mogadanez]

getting it for testing. at least works in positive scenario. =)

will check drop connections at night backup

[zhaohanweng]

beta3 works nicely with our stack, we use ELB in front of the Riak nodes. In 2.1.1 When we use .withMinConnections(10), we will have the following error when the ELB idle timeout happens:

error: [RiakConnection] (riak-elb.testurl.com:8087) Error: read ECONNRESET
...
error: [RiakConnection] (riak-elb.testurl.com:8087) Error: This socket has been ended by the other party

Now still using .withMinConnections(10) will not cause any errors after the ELB closes all the connections after the idle timeout.

[lukebakken]

@mogadanez - thank you, I look forward to your report.

@zhaohanweng - thank you for letting me know that beta3 fixes your issues.

[mogadanez]

@lukebakken awesome, looks like all is fine now, backup proceed smooth.

[alexmoore]

Tests pass locally, just thumbing through the code.

[alexmoore]

:+1:

[lukebakken]

@mogadanez, @zhaohanweng, @flohdot - I have just published beta4 with one small change to the re-try logic. The integration tests as well as a test using my chaos-monkey script and loader program work great.

If you have time to test beta4 in your environments, that would be great.

I expect to release beta4 as 2.1.2 this coming Monday, March 14th. Thanks!

$ npm dist-tag ls basho-riak-client
beta: 2.1.2-beta4
latest: 2.1.1

[flohdot]

Thanks! @zippunov, check it out. :)