Performance issue with SQS with sslEnabled: true

[runvnc]

Hello. I'm running into an issue with SQS taking quite a long time on my system when using sslEnabled: true. It seems like at least the createQueue calls are slow and also possibly the sendMessage. Here are some lines from my log.

with sslEnabled: false

Queue constructor returned for manager-runvnc-Rev-1-0 in 405 ms.
Queue constructor returned for testappmgr in 367 ms.
Request returned in 839 ms. Sending response to server..
Request returned in 1038 ms. Sending response to server..

with sslEnabled: true

Queue constructor returned for testappmgr in 12148 ms.
Request returned in 9527 ms. Sending response to server..
Request returned in 9524 ms. Sending response to server.

So those logs are measuring the time it takes for createQueue and also sendMessage.

First question I have, and I apologize if this is a bit ignorant, but I would like to just leave sslEnabled: false, but I'm not sure if that would be a security problem. If I have most of my code running on EC2, by any chance is there some aspect of Amazon's VM configuration that would mean that my HTTP messages for AWS API requests would be private, even if they weren't encrypted with SSL (doubt it, just thought I would ask).

Second question, is there something about my configuration (or code) that I can change/fix to get better performance from SQS with sslEnabled: true? (Maybe something to do with OpenSSL or something?)

Third question, is this perhaps a bug, or is there maybe a way that the AWS SDK for Node implementation could be improved to provide better performance with sslEnabled: true?

Thanks so much for your help.

[runvnc]

OK so I looked at the AWS Node code and came up with a hack:

aws.NodeHttpClient.sslAgent.options.rejectUnauthorized = false
aws.NodeHttpClient.sslAgent.maxSockets = 500

Not sure which one of those does it, probably the maxSockets, but when I use that the performance I see in my application is similar to with sslEnabled: false.

[lsegal]

@ithkuil have you tried with just maxSockets? Rejecting unauthorized certificates is a big part of guaranteeing security when using SSL.

[runvnc]

I tried turning the rejectunauthorized back on since you suggested it. I think it is a little bit slower now but not nearly as bad as before without the maxSockets.

Do you think it would make sense to make it so that the https options can be passed through to the ssl agent the way you can with the non-http?

[lsegal]

There are a couple of ways you can do this already without directly modifying the NodeHttpClient.sslAgent.

First, you can pass your own agent to httpOptions:

var https = require('https');
var agent = new https.Agent();
agent.maxSockets = 500;
agent.rejectUnauthorized = true;

AWS.config.httpOptions = {agent: agent};

Alternatively, you can configure the global HTTPS agent to accept more concurrent connections and tell the SDK to use that:

var https = require('https');
https.globalAgent.maxSockets = 500;
https.globalAgent.rejectUnauthorized = true;
AWS.config.httpOptions = {agent: https.globalAgent};

We don't use the globalAgent by default because in 0.8.x, the rejectUnauthorized property on that agent instance is set to false, which is not a secure setting. Node 0.10.x changes this, so in the future we may begin to use the globalAgent, and this would mean doing everything in the above except for having to set the httpOptions.

Do these options work for you?

[runvnc]

That's fine, thanks. I didn't realize I could pass my own agent in.

[brianc]

Just wanted to throw in my 2 cents - I stumbled across this issue googling for how to set maxSockets properly in the aws sdk. Doing what @lsegal suggested sped up my batching processing immensely. It's really unfortunate node has such a low default setting on that property. Might be worth while to publish the information on upping max sockets in the documentation somewhere. Thanks. :+1:

[dsjoerg]

Just got bit by this myself

[lock[bot]]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs and link to relevant comments in this thread.