AKS rate limitting

[illume]

It's been reported that Headlamp running against AKS can run up against API rate limiting.

Headlamp currently polls K8s APIs quite frequently.

• can the issue be reproduced?
• investigate possibilities to drop rate limiting on AKS
• looking into using event based APIs instead

[illume]

Here's an example function using the /watch Kubernetes part of the REST API.

See "Efficient detection of changes" in the K8s docs.

This is for watching events on a CR, but we can use it for other resources too.

/**
 * Watch for events on a Kubernetes custom resource using the REST API
 * 
 * @param baseUrl - The base URL of the Kubernetes API server
 * @param apiVersion - The API version to use (e.g. "v1", "v1beta1")
 * @param namespace - The namespace of the custom resource (optional)
 * @param resourceName - The name of the custom resource (e.g. "mycrd.example.com")
 * @param onEvent - The function to call when a new event is received
 */
async function watchCustomResource(
  baseUrl: string,
  apiVersion: string,
  namespace: string | undefined,
  resourceName: string,
  onEvent: (event: any) => void
) {
  const apiUrl = `${baseUrl}/apis/${apiVersion}/namespaces/${namespace}/` +
    `${resourceName}?watch=1`;

  while (true) {
    try {
      const response = await fetch(apiUrl);

      if (!response.ok) {
        throw new Error(`Failed to fetch resource: ${response.statusText}`);
      }

      const reader = response.body?.getReader();
      if (!reader) {
        throw new Error(`Failed to get response reader`);
      }

      let buffer = '';
      while (true) {
        const { done, value } = await reader.read();
        if (done) {
          break;
        }
        buffer += new TextDecoder().decode(value);
        const events = buffer.split('\n');
        buffer = events.pop() || '';

        for (const event of events) {
          if (event.length === 0) {
            continue;
          }
          const parsedEvent = JSON.parse(event);
          onEvent(parsedEvent);
        }
      }
    } catch (error) {
      console.error(`Error watching resource: ${error}`);
    }
    // Wait 5 seconds before attempting to reconnect
    await new Promise(resolve => setTimeout(resolve, 5000));
  }
}

// Here's how it could be used.
watchCustomResource(
  "https://my-kubernetes-cluster-api-server",
  "v1",
  "default",
  "mycustomresource",
  event => console.log(`Received event: ${JSON.stringify(event)}`)
);