InvalidWorkerCreation: worker boot error: failed to create the graph

[gianmarcocalbi]

Bug report

• [x] I confirm this is a bug with Supabase, not with my own application.
• [x] I confirm I have searched the Docs, GitHub Discussions, and Discord.

Describe the bug

When I serve a function locally that imports a file outside /supabase/functions I get the error:

{"code":"BOOT_ERROR","message":"Worker failed to boot (please check logs)"}

The error does not occur when launching the function with $ deno run.

Logs

serving the request with ***/supabase/functions/hello-world
failed to create the graph
An error has occured
InvalidWorkerCreation: worker boot error: failed to create the graph
    at async UserWorker.create (ext:sb_user_workers/user_workers.js:145:15)
    at async Object.handler (file:///root/index.ts:154:22)
    at async respond (ext:sb_core_main_js/js/http.js:163:14) {
  name: "InvalidWorkerCreation"
}

Reproduce steps

1. $ supabase functions new hello-world

2. Replace hello-world/index.ts with:

   // @deno-types="npm:@types/express@4.17.15"
   import express, { Request, Response } from 'npm:express@4.21.0';
   import { hello } from '../../../lib.ts';
   
   const app = express();
   app.use(express.json());
   
   const port = 3000;
   
   app.get('/hello-world', (_: Request, res: Response) => {
     res.send(hello());
   });
   
   app.listen(port, () => {});

3. create a file lib.ts in the project root containing the following:

   export function hello() : string {
     return 'Hello World!';
   }

4. run the function with supabase: $ supabase functions serve hello-world --no-verify-jwt the error occurs
5. run the function with deno (from within the /supabase/functions/hello-world folder): $ deno run -A index.ts no issues

[nyannyacha]

This issue seems to be caused by the CLI only mounting the /supabase/functions directory that exists on the host machine into the edge runtime container. From the container's perspective, it looks like no files exist in the parent directory, and as a result, lib.ts fails to load.

// docker inspect <edge runtime container id>
...
"Mounts": [
    {
        "Type": "volume",
        "Name": "supabase_edge_runtime_cli",
        "Source": "/var/lib/docker/volumes/supabase_edge_runtime_cli/_data",
        "Destination": "/root/.cache/deno",
        "Driver": "local",
        "Mode": "rw",
        "RW": true,
        "Propagation": ""
    },
    {
        "Type": "bind",
        "Source": "/Users/nyan/Documents/repository/cli/supabase/functions",
        "Destination": "/Users/nyan/Documents/repository/cli/supabase/functions",
        "Mode": "ro",
        "RW": false,
        "Propagation": "rprivate"
    }
],
...

serving the request with ***/supabase/functions/hello-world
failed to create the graph
An error has occured

And simply exposing a failed to create the graph message seems problematic. We actually have a source about why the graph failed to be created for some reason, but we didn't render it to the console or in the response.

Regarding the mount issue you're having, there's nothing the edge runtime can do to fix it, but the error message needs improvement, so I'll be submitting a PR soon.

--

Error message should be able to be improved as follows:

Response

{
    "msg": "InvalidWorkerCreation: worker boot error: failed to create the graph: Module not found \"file:///workspaces/lib.ts\".\n    at file:///workspaces/edge-runtime/examples/serve/index.ts:3:23"
}

Console

worker boot error: failed to create the graph: Module not found "file:///workspaces/lib.ts".
    at file:///workspaces/edge-runtime/examples/serve/index.ts:3:23    
InvalidWorkerCreation: worker boot error: failed to create the graph: Module not found "file:///workspaces/lib.ts".
    at file:///workspaces/edge-runtime/examples/serve/index.ts:3:23
    at async UserWorker.create (ext:sb_user_workers/user_workers.js:145:15)
    at async createWorker (file:///workspaces/edge-runtime/examples/main/index.ts:85:12)
    at async callWorker (file:///workspaces/edge-runtime/examples/main/index.ts:103:22)
    at async respond (ext:sb_core_main_js/js/http.js:163:14) {
  name: "InvalidWorkerCreation"
}

[gianmarcocalbi]

This issue seems to be caused by the CLI only mounting the /supabase/functions directory that exists on the host machine into the edge runtime container. From the container's perspective, it looks like no files exist in the parent directory, and as a result, lib.ts fails to load.

// docker inspect <edge runtime container id>
...
"Mounts": [
    {
        "Type": "volume",
        "Name": "supabase_edge_runtime_cli",
        "Source": "/var/lib/docker/volumes/supabase_edge_runtime_cli/_data",
        "Destination": "/root/.cache/deno",
        "Driver": "local",
        "Mode": "rw",
        "RW": true,
        "Propagation": ""
    },
    {
        "Type": "bind",
        "Source": "/Users/nyan/Documents/repository/cli/supabase/functions",
        "Destination": "/Users/nyan/Documents/repository/cli/supabase/functions",
        "Mode": "ro",
        "RW": false,
        "Propagation": "rprivate"
    }
],
...

serving the request with ***/supabase/functions/hello-world
failed to create the graph
An error has occured

And simply exposing a failed to create the graph message seems problematic. We actually have a source about why the graph failed to be created for some reason, but we didn't render it to the console or in the response.

Regarding the mount issue you're having, there's nothing the edge runtime can do to fix it, but the error message needs improvement, so I'll be submitting a PR soon.

--

Error message should be able to be improved as follows:

Response

{
    "msg": "InvalidWorkerCreation: worker boot error: failed to create the graph: Module not found \"file:///workspaces/lib.ts\".\n    at file:///workspaces/edge-runtime/examples/serve/index.ts:3:23"
}

Console

worker boot error: failed to create the graph: Module not found "file:///workspaces/lib.ts".
    at file:///workspaces/edge-runtime/examples/serve/index.ts:3:23    
InvalidWorkerCreation: worker boot error: failed to create the graph: Module not found "file:///workspaces/lib.ts".
    at file:///workspaces/edge-runtime/examples/serve/index.ts:3:23
    at async UserWorker.create (ext:sb_user_workers/user_workers.js:145:15)
    at async createWorker (file:///workspaces/edge-runtime/examples/main/index.ts:85:12)
    at async callWorker (file:///workspaces/edge-runtime/examples/main/index.ts:103:22)
    at async respond (ext:sb_core_main_js/js/http.js:163:14) {
  name: "InvalidWorkerCreation"
}

Than you for the very detailed response.

I was hoping I could use some files outside the supabase/functions folder containing some utilities that are shared with the frontend. So I guess the solution is to keep these files into supabase/functions or to publish them as package and import them in a standard way, right?

[nyannyacha]

Yeah, it should because supabase/functions is the mount point of its container. All subpaths for this path should be recognizable from edge runtime.

[codinronan]

@nyannyacha any suggestions on debugging this error until such time as an update to the edge-runtime container is released? I am running into this, have gone through every single file (of over 1000, painful) and every single one has imports that are either in the imports list in deno.jsonc, or are contained within the supabase/functions folder. I'm probably missing one... So I need to see what the actual resolver error is but I can't figure it out. Any tips?

[nyannyacha]

Well...it's complicated, but it's possible...if you're good with Docker.

Finding the file in issue is possible with strace. You can access the shell of a container running edge runtime via docker exec -it <edge runtime container id> /bin/bash.

From there, you need to install the necessary packages for tracing by typing apt-get update && apt-get install -y htop strace.

You should be able to find the pid of the edge runtime using htop, which will allow you to observe the syscalls that the edge runtime is calling in real-time via strace.

After you type strace, as shown below, the shell will log syscalls until you press Ctrl+C.

Now send a request and observe the logs that are written to the shell. If you've followed these instructions well, you should have a syscall logged in your shell that failed to open the file, like this.

I hope you find these instructions helpful. Have a great day!

[codinronan]

@nyannyacha Thank you I will give that a try!

[codinronan]

@nyannyacha I was able to successfully execute the steps above, however it turned out my error wasn't a missing file, it was the following deno error:

You'll note in the image it seems you've done some work to spit out the real error :) So it's nice to see that now, but for yesterday, that's what I was running into, across several imports in fact.

Some needed to be re-spec'd in the import_map.json file (removed the slashes and replaced with dashes) so it wouldn't think they were paths, and I also needed to add the --import-map cli parameter to most of my commands.

Clearly some pretty deep deno changes going on here. I can imagine it's been rough for you guys keeping up with them. I'm glad the supabase cli provides these breakouts like the --import-map parameter to fix things otherwise we'd be pretty screwed.

So for my purposes I'm good here, hopefully the above steps will help someone else!

[nyannyacha]

Yeah in such a case like yours, the syscall log would look normal, so the method I suggested instructions above would not have been able to resolve the issue.

There are many other cases where a graph error is emitted besides the file not being found, so I will submit a patch in the near future to expose the detailed error.

[onyedikachi23]

This issue seems to be caused by the CLI only mounting the /supabase/functions directory that exists on the host machine into the edge runtime container. From the container's perspective, it looks like no files exist in the parent directory, and as a result, lib.ts fails to load.

// docker inspect <edge runtime container id>
...
"Mounts": [
    {
        "Type": "volume",
        "Name": "supabase_edge_runtime_cli",
        "Source": "/var/lib/docker/volumes/supabase_edge_runtime_cli/_data",
        "Destination": "/root/.cache/deno",
        "Driver": "local",
        "Mode": "rw",
        "RW": true,
        "Propagation": ""
    },
    {
        "Type": "bind",
        "Source": "/Users/nyan/Documents/repository/cli/supabase/functions",
        "Destination": "/Users/nyan/Documents/repository/cli/supabase/functions",
        "Mode": "ro",
        "RW": false,
        "Propagation": "rprivate"
    }
],
...

serving the request with ***/supabase/functions/hello-world
failed to create the graph
An error has occured

And simply exposing a failed to create the graph message seems problematic. We actually have a source about why the graph failed to be created for some reason, but we didn't render it to the console or in the response.

Regarding the mount issue you're having, there's nothing the edge runtime can do to fix it, but the error message needs improvement, so I'll be submitting a PR soon.

--

Error message should be able to be improved as follows:

Response

{
    "msg": "InvalidWorkerCreation: worker boot error: failed to create the graph: Module not found \"file:///workspaces/lib.ts\".\n    at file:///workspaces/edge-runtime/examples/serve/index.ts:3:23"
}

Console

worker boot error: failed to create the graph: Module not found "file:///workspaces/lib.ts".
    at file:///workspaces/edge-runtime/examples/serve/index.ts:3:23    
InvalidWorkerCreation: worker boot error: failed to create the graph: Module not found "file:///workspaces/lib.ts".
    at file:///workspaces/edge-runtime/examples/serve/index.ts:3:23
    at async UserWorker.create (ext:sb_user_workers/user_workers.js:145:15)
    at async createWorker (file:///workspaces/edge-runtime/examples/main/index.ts:85:12)
    at async callWorker (file:///workspaces/edge-runtime/examples/main/index.ts:103:22)
    at async respond (ext:sb_core_main_js/js/http.js:163:14) {
  name: "InvalidWorkerCreation"
}

@nyannyacha I'm having a very similar problem here, but I'm not using an Express server.

This is sub-directory setup under my supabase directory:

I don't really know how Deno and Docker works and from this your reply I qouted, I don't know where to start debugging, so I might have to paste the codes in the sub directories here:

• In /_utils I have stripe.ts:

     // esm.sh is used to compile stripe-node to be compatible with ES modules.
import Stripe from 'https://esm.sh/stripe@16.12.0?target=deno&deno-std=0.132.0&no-check';

export const stripe = Stripe(Deno.env.get('STRIPE_SECRET_KEY') ?? '', {
  httpClient: Stripe.createFetchHttpClient(),
});import { get } from 'react-native/Libraries/TurboModule/TurboModuleRegistry';

• in /payment-sheet I have index.ts:

// Follow this setup guide to integrate the Deno language server with your editor:
// https://deno.land/manual/getting_started/setup_your_environment
// This enables autocomplete, go to definition, etc.

// Setup type definitions for built-in Supabase Runtime APIs 
import "jsr:@supabase/functions-js/edge-runtime.d.ts";
import { stripe } from "../_utils/stripe.ts";

console.log("Hello from Functions!");

Deno.serve(async (req) => {

  try {
    const { amount } = await req.json();

  const paymentIntent = await stripe.paymentIntents.create({
    amount: 1099,
    currency: "usd"
  });
    console.log(paymentIntent);

  const res = {
    paymentIntent: paymentIntent.client_secret,
    publishabledKey: Deno.env.get("EXPO_PUBLIC_STRIPE_PUBLISHABLE_KEY")
  };

  return new Response(
    JSON.stringify(res), 
    { headers: { "Content-Type": "application/json" } },
  );

  } catch (error) {
    console.log(error);

    return new Response(JSON.stringify(error), {
      headers: { "Content-Type": "application/json" },
      status: 400,
    })
  }

});

/* To invoke locally:

  1. Run `supabase start` (see: https://supabase.com/docs/reference/cli/supabase-start)
  2. Make an HTTP request:

  */

  /* curl -i --location --request POST 'http://127.0.0.1:54321/functions/v1/payment-sheet' \
    --header 'Authorization: Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJzdXBhYmFzZS1kZW1vIiwicm9sZSI6ImFub24iLCJleHAiOjE5ODM4MTI5OTZ9.CRXP1A7WOeoJeXxjNni43kdQwgnWNReilDMblYTn_I0' \
    --header 'Content-Type: application/json' \
    --data '{"amount":1150}'
 */

You can see the curl command I'm using thats giving the error whenever I run it.

For further info, I'm following a tutorial, heres the GitHub repo: food-ordering-app

[codinronan]

@onyedikachi23 have you also run supabase functions serve from inside your src directory? (looks like src from your screenshot, has to be one folder above the supabase folder).

[onyedikachi23]

@codinronan I run the command from the root directory (and yes, /src is one level above the /supabase):

npx supabase functions serve --env-file .env payment-sheet

[hitamu]

I'm facing the same issue. Previously it was working fine until I change my folder name of the edge function

[onyedikachi23]

@hitamu can I see how your setup was when it was working fine? I need to try and see if that helps me out.

[github-actions[bot]]

:tada: This issue has been resolved in version 1.58.11 :tada:

The release is available on GitHub release

Your semantic-release bot :package::rocket:

[github-actions[bot]]

:tada: This issue has been resolved in version 1.58.11 :tada:

The release is available on GitHub release

Your semantic-release bot :package::rocket:

[junedkhatri31]

How to use version 1.58.11 in local supabase?

[codinronan]

How to use version 1.58.11 in local supabase?

I don't think it's possible without action from the supabase team to make it available in the cli, but hoping I'm wrong and they reply because I'd like to try it as well.

[junedkhatri31]

@codinronan I built my own cli from source - https://github.com/supabase/cli/

[nyannyacha]

@junedkhatri31 @codinronan

https://github.com/supabase/cli/releases/tag/v1.202.3 https://www.npmjs.com/package/supabase?activeTab=versions 😋

[onyedikachi23]

How do I update the Supabase Edge Runtime in the Expo project?

I'm following this guide at supabase/edge-runtime , so is running this enough:

docker pull ghcr.io/supabase/edge-runtime:v1.58.11