Highly inconsistent isolate start times

[cknight]

Problem description

This isn't necessarily a bug report per se, but isolate start times seem slower than previously and much more variable.

What factors affect the isolate start time in Deploy? For example, in the last 20 minutes or so, isolate start times in my global-db-comparison project are varying wildly from blazing fast (144 ms) to very slow (2.2 sec). Can we expect more consistent, faster, times in the future?

Steps to reproduce

N/A

Expected behavior

This is more desired than expected and I do not know the limitations you are working under, but ideally:

• Isolates start times do not vary dramatically and are generally consistent
• Isolates start times are less than 200ms

Environment

No response

Possible solution

No response

Additional context

No response

[AW0005]

I'm experiencing similar, and I rarely get under 1s isolate start times. Is this something that upgrading from the free tier would change or is this just the way deno deploy is right now?

[cknight]

In another project (dev-tools), I am seeing wildly different times (from 144ms to 9.14s). This is a simple Fresh project with very little going on (primarily client side JS), and no special imports, KV, WASM, broadcast channel or other special features employed.

[dbushell]

I'm also experiencing very slow start up times.

I rarely see below 100ms and more often 500ms+ to over a second.

This defeats the point of "edge" hosting and makes Deno Deploy not a viable platform for professional use.

[patrickalima98]

I'm also experiencing very slow start up times. But for my case, i verified the problem is with the Import modules.

In this example, I have a global file with 4x imports, and its run with 2seg only for import files on deno deploy:

[patrickalima98]

More updates, its very estrange how it is random:

This problem its not 100% for all locations. For example the first request of the day for us-east-4 and europe in general is acceptable. The main problem is in the south america

[igorzi]

FYI - this is being investigated by the Deploy team.

[patrickalima98]

Thanks for comment @igorzi, only to put here more details, a little and very simple script is having long cold start too:

[patrickalima98]

Hi, to help provide some more use cases I'm attaching here my code where I'm using it for production and having 6 seconds of cold start every now and then, most of the time it works fine with an average of 250 - 400ms:

import { config } from 'npm:dotenv';
import { Hono } from "hono/preset/quick.ts";
config();

const app = new Hono();

app.onError((err: any, c): any => {
  if (err.message === 'Validation failure') {
    return c.json({ errors: err.messages }, err.status)
  }

  return c.json(err.error, err.status)
})

const routes = {
  user: 'name_of_a_deno_project',
}

app.all('*', (c) => {
  const url = new URL(c.req.url);
  const { pathname, search } = url;
  const pathWithParams = pathname + search;

  const foward2 = url.pathname.match(new RegExp(Object.keys(routes).join('|')));
  if (foward2 && foward2.length) {
    const newPath = `https://${routes[foward2[0] as keyof typeof routes]}-${Deno.env.get('ENV_TYPE')}.deno.dev` ;
    console.info('newPath ', newPath + pathWithParams)
    const newReq = new Request(newPath + pathWithParams, c.req.raw);

    return fetch(newReq);
  }

  return c.text('');
})

Deno.serve(app.fetch);

The cold start:

[algoflows]

@igorzi no feedback from the team?? I didn't see any resolution from anyone on the threads here. I am about to buy into Deno for the long haul but this is putting a huge big red flag stop on things. I left some questions in the Deno Discord, the only response I got was to message an email address with my question. I've got a super bad feeling about the deno project. I think I might give it a swerve.

[arnauorriols]

@algoflows Thanks for surfacing this thread. Coldstarts were an issue we worked on intensively last quarter, with significant improvements. I'll try to put together some data to back this announcement and offer a more detailed answer tomorrow 🙏

Meanwhile, maybe some of you that have experienced this issue in the past could give some feedback on what's the status on your end?

[cknight]

I can confirm that I see much more consistent start times. In an old fresh project, these times are between 300ms-700ms with the odd outliers at 1s. For a non-Fresh project times are much snappier at 40ms (!!) - 200ms with the odd outlier up to 450ms.

Nothing is in the logs over 1.5s, which was definitely not the case when I raised this issue.

[patrickalima98]

In my case, it's very similar to @cknight's feedback. No more than 1.5s... But I think this is still too high for a cold start. But for a full CRUD project with lots of routes, I'm getting between 400ms and 1.5ms.

The place where there is always a bigger cold start is in South America:

This is from a simple Hello World:

[dbushell]

My status: I moved most projects off of Deno Deploy due to the issue. The lack of updates left me with no confidence. I don't have hard data but it always felt slow. The more extreme 1+ second times were less frequent at least.