statsig-io / node-js-server-sdk

Statsig's SDK for server-side Node.js applications.
ISC License
20 stars 15 forks source link

ID Lists timeout making the server to crash #20

Closed kaioduarte closed 1 year ago

kaioduarte commented 1 year ago

We are facing random timeouts when updating the ID Lists, and as they are unhandled exceptions it makes the server instance die.

 FetchError: request to https://idliststorage.blob.core.windows.net/idlists/<omitted_id>?sv=2020-10-02&se=2022-12-04T21%3A39%3A36Z&sr=b&sp=r&sig=<omitted_id> failed, reason: connect ETIMEDOUT 20.150.78.68:443
    at ClientRequest.<anonymous> (/app/node_modules/node-fetch/lib/index.js:1491:11)
    at ClientRequest.emit (node:events:513:28)
    at TLSSocket.socketErrorListener (node:_http_client:494:9)
    at TLSSocket.emit (node:events:513:28)
    at emitErrorNT (node:internal/streams/destroy:157:8)
    at emitErrorCloseNT (node:internal/streams/destroy:122:3)
    at processTicksAndRejections (node:internal/process/task_queues:83:21) { 

Any advice? Or should it be handled on the lib side and only logged as an error?

tore-statsig commented 1 year ago

Thanks for the report, this should be handled by the sdk itself and we are investigating

daniel-statsig commented 1 year ago

Hey @kaioduarte, I ran this locally and was able to get the exact same error, but these are caught and simply logged as a warning. The server should still be running after this.

Here is the exact line where this is getting printed out https://github.com/statsig-io/node-js-server-sdk/blob/main/src/SpecStore.ts#L477

We will retry to sync the id lists, so this is only really a warning.

Do you have more context that could point to this actually crashing the node process?

kaioduarte commented 1 year ago

Hi @daniel-statsig! Thanks for pointing out the source of the error, so it seems unrelated to the issue we have. GCP makes console.error output too prominent in the errors list, which misleads us sometimes.

I'm closing the issue, thanks again!

daniel-statsig commented 1 year ago

Sorry for the confusion, I'll update it to console.warn, since that better reflects what this is. Good luck on finding the real cause.