search

salesforce-marketingcloud / FuelSDK-Node

BSD 3-Clause "New" or "Revised" License
49 stars 24 forks source link

Connection exceptions after upgrade #10

Open WithSecureSamiTikka opened 7 years ago

WithSecureSamiTikka commented 7 years ago

We run AWS Lambda function which uses FuelSDK to update a Data Extension. We recently upgraded the SDK from 1.1.8 to 2.1.0. After the upgrade the lambda function started crashing with following errors

START RequestId: 7adcc610-33b7-4c1d-9ebe-38ad8957525e Version: 37
2016-12-03T04:56:14.799Z    4722357d-fda6-49d0-87df-28538e923b27    Error: connect ECONNRESET 198.245.89.89:443
at Object.exports._errnoException (util.js:870:11)
at exports._exceptionWithHostPort (util.js:893:20)
at TCPConnectWrap.afterConnect [as oncomplete] (net.js:1062:14)
END RequestId: 7adcc610-33b7-4c1d-9ebe-38ad8957525e
REPORT RequestId: 7adcc610-33b7-4c1d-9ebe-38ad8957525e  Duration: 35.29 ms  Billed Duration: 100 ms Memory Size: 256 MB Max Memory Used: 49 MB  
Process exited before completing request

Also

START RequestId: 30200f4c-20df-49e7-9876-b18635d7dd94 Version: 37
2016-12-03T04:56:59.816Z    396a31ac-1671-46ad-818f-61f9b8597cc3    Error: socket hang up
at createHangUpError (_http_client.js:200:15)
at TLSSocket.socketOnEnd (_http_client.js:285:23)
at emitNone (events.js:72:20)
at TLSSocket.emit (events.js:166:7)
at endReadableNT (_stream_readable.js:905:12)
at nextTickCallbackWith2Args (node.js:437:9)
at process._tickDomainCallback (node.js:392:17)
END RequestId: 30200f4c-20df-49e7-9876-b18635d7dd94
REPORT RequestId: 30200f4c-20df-49e7-9876-b18635d7dd94  Duration: 34.86 ms  Billed Duration: 100 ms Memory Size: 256 MB Max Memory Used: 52 MB  
Process exited before completing request

These errors seem to mostly happen during slow traffic hours and before the error there has been 1-2 minutes of idle time. My hypothesis is the Exact Target server side has closed the socket and because that is not handled, it causes an exception which kills the process.

Has something changed in the SDK because this problem started happening after the SDK upgrade?

According to the logs, the exception occurs before any of our code runs in the lambda function, making it impossible to put a try+catch around it. Would it be possible to either:

sjakthol commented 7 years ago

It appears that the errors are caused by an extra call to the API performed in the background whenever the SOAP api is used: https://github.com/salesforce-marketingcloud/FuelSDK-Node/blob/master/lib/ET_Client.js#L79-L86

That request happens outside all normal control flow and no one is handling the response. If that request fails for one reason or another (in this case the Lambda function container was frozen for some minutes while the request was still in-flight), the error will be converted into an uncaught exception and causing the process to crash.

Git history reveals that this code was added in PR #6. @amagar088, what was the reason for adding this extra request to the library? Could this phone-home functionality be implemented in some other way as part of the normal processing flow so that errors from that can be handled by the application?