LaunchDarkly occasionally erroring on Vercel

[IGassmann]

Describe the bug We're occasionally observing the following error on our Next.js application deployed to Vercel:

warn: [LaunchDarkly] Received I/O error (Client network socket disconnected before secure TLS connection was established) for polling request - will retry

To reproduce Steps to reproduce the behavior.

1. Set up a Next.js app
2. Install @launchdarkly/node-server-sdk@9.0.3
3. Set up LaunchDarkly as follows:

import { init, type LDClient } from '@launchdarkly/node-server-sdk';

let launchDarklyClient: LDClient;

async function initialize() {
  const launchDarklySDKKey = process.env.LAUNCH_DARKLY_SDK_KEY;
  if (!launchDarklySDKKey) {
    throw new Error('LAUNCH_DARKLY_SDK_KEY environment variable is not set.');
  }
  const client = init(launchDarklySDKKey, { stream: false });
  await client.waitForInitialization();
  return client;
}

export async function getLaunchDarklyClient(): Promise<LDClient> {
  if (launchDarklyClient) return launchDarklyClient;
  return (launchDarklyClient = await initialize());
}

4. Use a server-side feature flag within an endpoint that uses getLaunchDarklyClient().

Expected behavior We shouldn't see the above error on Vercel logs.

Logs

warn: [LaunchDarkly] Received I/O error (Client network socket disconnected before secure TLS connection was established) for polling request - will retry

SDK version @launchdarkly/node-server-sdk: 9.0.3

Language version, developer tools "next": "14.0.2" "node": "18.x"

OS/platform Vercel

Additional context The code runs on Vercel Serverless functions, which utilize AWS Lambda behind the scenes. I wonder if this issue is happening because we're initializing the SDK within the handler instead of within the serverless function execution context. The LaunchDarkly docs aren't clear on what strategy we should use. In the Next.js code example, it shows it being done within the handler, but in the Serverless code example it is done outside the handler.

[kinyoklion]

Hello @IGassmann,

What is the frequency of occasionally? It is expected that there are network errors sometimes, and sometimes those will be I/O errors. Either a connection drop, or in this case a connection drop while attempting to establish a connection. As noted it will retry.

Where you initialize it would be based on how the specific runtime environment works, as long as you are able to make outgoing network requests before the handler executes, then doing the initialization outside the handler should be fine.

Thanks, Ryan

[IGassmann]

Hey @kinyoklion,

What is the frequency of occasionally?

Around ~2 errors for every 2,000 requests.

Where you initialize it would be based on how the specific runtime environment works, as long as you are able to make outgoing network requests before the handler executes, then doing the initialization outside the handler should be fine.

Next.js on Vercel uses AWS Lamba to run server-side code. Does that mean the following is preferable over what we have above:

 import { init, type LDClient } from '@launchdarkly/node-server-sdk';

const launchDarklySDKKey = process.env.LAUNCH_DARKLY_SDK_KEY;
if (!launchDarklySDKKey) {
  throw new Error('LAUNCH_DARKLY_SDK_KEY environment variable is not set.');
}

const launchDarklyClient = init(launchDarklySDKKey, { stream: false });

export async function getLaunchDarklyClient(): Promise<LDClient> {
  await client.waitForInitialization();
  return launchDarklyClient;
}

Why is that?

[kinyoklion]

The earlier the client can be initialized the less waiting once a handler actually starts executing. I don't know that it would be any more desirable for your case, or have any impact on how often you are getting IO errors.

The code from the next.js sample has a bug, so we will get that updated. If you call getLaunchDarklyClient() several times, then can initialize several clients. This is because the client is not assigned to the global until after the await. (@mmrj We need to take a look at that sample)

async function initialize() {
  const launchDarklySDKKey = process.env.LAUNCH_DARKLY_SDK_KEY;
  if (!launchDarklySDKKey) {
    throw new Error('LAUNCH_DARKLY_SDK_KEY environment variable is not set.');
  }
  const client = init(launchDarklySDKKey, { stream: false });

  await client.waitForInitialization();
  // CALLING getLaunchDarklyClient between these two lines will result in multiple clients
  // getting initalized. After we start to await, but before the await resolves. 
  return client;
}

export async function getLaunchDarklyClient(): Promise<LDClient> {
  if (launchDarklyClient) return launchDarklyClient;
  return (launchDarklyClient = await initialize());
}

A better approach, if you needed to wait for initialization it to set the global with the return value from init. Then do the waitForInitialization in the getLaunchDarklyClient. You can call that function safely multiple times, and it means no matter how many outstanding requests you have they resolve the same client once it is ready.

I am not very familiar with Next.js, so specific things may make this situation not happen, but from a purely JS perspective it is problematic.

None of this should really affect your IO situation, aside from potentially changing the number of clients that attempt to be initialized.

Thanks, Ryan

[IGassmann]

Hey @kinyoklion!

We updated the code to the following, but that didn't fix the issue. We're still frequently getting the same error.

import { init, type LDClient } from '@launchdarkly/node-server-sdk';

let launchDarklyClient: LDClient;

function initialize() {
  const launchDarklySDKKey = process.env.LAUNCH_DARKLY_SDK_KEY;
  if (!launchDarklySDKKey) {
    throw new Error('LAUNCH_DARKLY_SDK_KEY environment variable is not set.');
  }
  launchDarklyClient = init(launchDarklySDKKey, { stream: false });
}

export async function getLaunchDarklyClient(): Promise<LDClient> {
  if (!launchDarklyClient) {
    initialize();
  }

  await launchDarklyClient.waitForInitialization();
  return launchDarklyClient;
}

[kinyoklion]

Hello @IGassmann,

I don't really think there is anything else we can do SDK side. The most likely underlying cause for this is a network related issue. Either with the host itself, some intermediate infrastructure, etc.

You can contact LaunchDarkly support and they may be able to assist. Let them know about this thread, so they know we have looked into SDK issues already.

You may also be able to contact Vercel support to check if they are aware of any issues.

Thank you, Ryan

[AbelGuitian]

@IGassmann Did you fixed this? I'm having same problem.

[IGassmann]

@AbelGuitian I never was able to fix it :/