Allow creating isolated pool for each request with global runtime in CF Workers

[NathanFlurry]

Motivation

CF workers requires an isolated pool for every request. In order to achieve this, we currently create a new runtime for every request. We can speed up requests by creating a new runtime once and reusing it.

[linear[bot]]

OGBE-159 Allow creating isolated pool for each request with global runtime in CF Workers

[greptile-apps[bot]]

To allow creating an isolated pool for each request with a global runtime in CF Workers, follow these steps:

1. Modify server.ts to reuse the global runtime:

import { Runtime } from "./runtime.ts";
import { createWorkerPool, runJob, shutdownPool } from "../utils/worker_pool.ts";

const globalRuntime = new Runtime();

export async function handleRequest<DependenciesSnakeT, DependenciesCamelT, ActorsSnakeT, ActorsCamelT>(
  req: Request,
  info: RequestInfo,
): Promise<Response> {
  const url = new URL(req.url);

  // Handle CORS preflight
  if (req.method === "OPTIONS") {
    return globalRuntime.corsPreflight(req);
  }

  // Disallow even simple requests if CORS is not allowed
  if (!globalRuntime.corsAllowed(req)) {
    return new Response(undefined, {
      status: 403,
      headers: {
        "Vary": "Origin",
        ...globalRuntime.corsHeaders(req),
      },
    });
  }

  // Only allow POST requests
  if (req.method !== "POST") {
    return new Response(undefined, {
      status: 405,
      headers: {
        "Allow": "POST",
        ...globalRuntime.corsHeaders(req),
      },
    });
  }

  // Get module and script name
  const matches = MODULE_CALL.exec(url.pathname);
  if (!matches?.groups) {
    return new Response(
      JSON.stringify({
        "message": "Route not found. Make sure the URL and method are correct.",
      }),
      {
        headers: {
          "Content-Type": "application/json",
          ...globalRuntime.corsHeaders(req),
        },
        status: 404,
      },
    );
  }

  // Lookup script
  const moduleName = matches.groups.module;
  const scriptName = matches.groups.script;
  const script = globalRuntime.config.modules[moduleName]?.scripts[scriptName];

  // Confirm script exists and is public
  if (!script || !script.public) {
    return new Response(
      JSON.stringify({
        "message": "Route not found. Make sure the URL and method are correct.",
      }),
      {
        headers: {
          "Content-Type": "application/json",
          ...globalRuntime.corsHeaders(req),
        },
        status: 404,
      },
    );
  }

  // Create context
  const ctx = globalRuntime.createRootContext({
    httpRequest: {
      method: req.method,
      path: url.pathname,
      remoteAddress: info.remoteAddress,
      headers: Object.fromEntries(req.headers.entries()),
    },
  });

  // Parse body
  let body;
  try {
    body = await req.json();
  } catch {
    const output = {
      message: "Request must have a valid JSON body.",
    };
    return new Response(JSON.stringify(output), {
      status: 400,
      headers: {
        "Content-Type": "application/json",
        ...globalRuntime.corsHeaders(req),
      },
    });
  }

  try {
    // Create isolated worker pool for the request
    const pool = createWorkerPool({ source: "./worker.ts" });
    const output = await runJob({ pool, request: body });
    shutdownPool(pool);

    if (output.__tempPleaseSeeOGBE3_NoData) {
      return new Response(undefined, {
        status: 204,
        headers: {
          ...globalRuntime.corsHeaders(req),
        },
      });
    }

    return new Response(JSON.stringify(output), {
      status: 200,
      headers: {
        "Content-Type": "application/json",
        ...globalRuntime.corsHeaders(req),
      },
    });
  } catch (e) {
    // Error response
    const output = {
      message: e.message,
    };

    return new Response(JSON.stringify(output), {
      status: 500,
      headers: {
        "Content-Type": "application/json",
        ...globalRuntime.corsHeaders(req),
      },
    });
  }
}

2. Ensure worker_pool.ts supports creating and managing isolated pools:

No changes needed in worker_pool.ts as it already supports creating and managing worker pools.

References

/src/runtime/server.ts /src/utils/worker_pool.ts /src/dynamic /src/runtime

Ask Greptile