Support continuation/async local storage

[matthewadams]

This issue is basically requesting the deno equivalent of https://www.npmjs.com/package/@northscaler/continuation-local-storage or https://nodejs.org/dist/latest-v14.x/docs/api/async_hooks.html#async_hooks_class_asynclocalstorage, with the additional feature as described at https://github.com/nodejs/node/issues/32062#issuecomment-614242505.

Without this, patterns like aspect-oriented programming (AOP) would be extremely difficult to implement. Here's an example of an AOP implementation based on @northscaler/aspectify that depends on continuation local storage to pull off its task: https://www.npmjs.com/package/@northscaler/securable-trait#security-via-aspect-oriented-programming-aop

[MarkTiedemann]

Not sure I understand your feature request. Can you sketch the API?

I noticed that you created the @northscaler/continuation-local-storage module. Why do you want this module (or something similar) to be a part of Deno (I presume, the standard library) rather than a third-party module?

[matthewadams]

Not sure I understand your feature request. Can you sketch the API?

The API would be similar to AsyncLocalStorage from Node.js: https://nodejs.org/dist/latest-v14.x/docs/api/async_hooks.html#async_hooks_class_asynclocalstorage. No need to sketch the API here. Just steal take inspiration from that.

I noticed that you created the @northscaler/continuation-local-storage module. Why do you want this module (or something similar) to be a part of Deno (I presume, the standard library) rather than a third-party module?

It could be a third-party module, it could be part of Deno. In any case, the primitives upon which the module would be based must be there: the deno equivalent of async_hooks. I can assure you that if AsyncLocalStorage is part of Deno, developers are going to have more confidence in its implementation. If it's not part of Deno but published as a third-party module from the core Deno team, that would also be favorable. I suppose that the the Deno async_hooks equivalent should be part of Deno itself, and the module that end users consume could be a third-party module or part of Deno. Should there be two issues to track here, then? One for the Deno equivalent of async_hooks and this one for the Deno equivalent of AsyncLocalStorage?

In any case, having this library is a key enabling technology for use cases like the AOP one I referenced above. IMHO, AsyncLocalStorage is as important to me in Deno as ThreadLocal is to Java, and deserves the consideration of being included in the core platform.

@ry Can you weigh in here?

[ry]

I totally missed the news on AsyncLocalStorage. Looks very useful.

I worry a lot about adding browser incompatible APIs. Is there any hope of doing AsyncLocalStorage in the browser?

[caspervonb]

Browser wise there's the https://github.com/WICG/kv-storage proposal which is kinda dead with no vendor interest.

[ry]

I don't think that's related AFAICT @caspervonb - it's just a typical kv storage not a "green thread local storage" or however you want to describe AsyncLocalStorage.

The "LocalStoarge" part of "AsyncLocalStorage" is very confusing. They are rather distinct APIs and shouldn't be using the same name.

[caspervonb]

Oh this is about TLS...

Apologies, didn't click through 😅

[vdeturckheim]

@ry I believe there is an initiative to revive zones but it might take a while before going there. If there is still a plan to support all Node.js stable APIs in Deno, one strategy could be to wait for AsyncLocaStorage to become stable and port it then. I plan to open a PR to move this to stable (in node) in the next couple months.

[matthewadams]

The "LocalStoarge" part of "AsyncLocalStorage" is very confusing. They are rather distinct APIs and shouldn't be using the same name.

@ry Yeah, I'm used to the Java name ThreadLocal. I'd be happy if Deno called it AsyncLocal or ContinuationLocal, personally. I will use the term AsyncLocal to mean Deno's version of Node.js's AsyncLocalStorage for the rest of this comment.

@vdeturckheim I'd ask you to not wait until AsyncLocalStorage is stable in Node.js before porting. You could keep Deno's AsyncLocal in experimental status the whole time so that we can use it & provide feedback. Then, after AsyncLocalStorage goes stable, you can later make AsyncLocal stable in Deno. One thing, though: make sure you're considering the ease-of-use ideas presented & being discussed at https://github.com/nodejs/node/issues/32062#issuecomment-614242505 and below.

[xialvjun]

I totally missed the news on AsyncLocalStorage. Looks very useful.

I worry a lot about adding browser incompatible APIs. Is there any hope of doing AsyncLocalStorage in the browser?

@ry https://www.npmjs.com/package/zone.js is for browser.

What's a zone? A Zone is an execution context that persists across async tasks. You can think of it as thread-local storage for JavaScript VMs.

Another issue about this is https://github.com/denoland/deno/issues/5638 . It seems you didn't participate in that issue, so I @ you.

[kitsonk]

Zones are dead as a TC39 proposal. There maybe an initial to revive it, but if anything it will be a cut down version of what was originally proposed, and therefore totally incompatible with what currently exists. Therefore there really isn't anything suitable that is standards based we can consider.

[xialvjun]

There is a PR https://github.com/denoland/deno/pull/8209 , but it's not for browser.

[winston0410]

Is there any update on this feature?

[kitsonk]

No. As stated clearly above, there is not a web standard that would introduce this, and it depends upon a withdrawn TC39 proposal. Given that Node.js async_hooks is still experimental, there is no web standard for this whole domain of stuff and there are a lack of underlying standard JavaScript APIs to implement it, it isn't going to happen any time soon.

Just commenting "any updates" is annoying, because it requires the maintainers to check out and see if there was actually some important information that was missed. If you want to voice support for something, add your 👍 to the top of the issue.

[zkulbeda]

Node.js AsyncLocalStorage is stable with run and getStore functions. docs

const storage = new AsyncLocalStorage();

function logWithId(msg) {
  const id = storage.getStore();
  console.log(`${id !== undefined ? id : '-'}:`, msg);
}

const result = await storage.run(crypto.randomUUID(), async () => {
  logWithId('start');
  await new Promise((resolve) => {
    setTimeout(() => {
      logWithId('finish');
      resolve();
    }, 200);
  });
  return "ok";
});

// result === "ok"
// storage.getStore() === undefined

[tom-sherman]

There is an async context proposal but it's pre Stage 1

https://github.com/legendecas/proposal-async-context

It also appears that Cloudflare will end up shipping a similar API in workers

https://github.com/cloudflare/workerd/pull/208

[benjamn]

I just opened https://github.com/legendecas/proposal-async-context/pull/17 over at the AsyncContext proposal repository, sharing my Deno-based native implementation of the current proposal.

If you have Docker installed, you can docker pull benjamn/deno:async-context (a mere 162MB, thanks to Deno being a standalone static binary!) and then drop yourself into a Deno REPL where AsyncContext is globally defined:

docker run -it --rm benjamn/deno:async-context repl

I would encourage anyone who has commented here to copy/paste this code into the REPL:

const c = new AsyncContext();
c.run(123, async () => {
  console.log("before await", c.get());
  const awaitResult = await new Promise(resolve => {
    setTimeout(() => resolve(c.get() + 111), 20);
  });
  console.log("after await", c.get(), { awaitResult });
})

Please note, I am not making any recommendation that AsyncContext should become an official part of Deno, but I figured I could bring new focus to this discussion by sharing a toy implementation.

If you think I might have messed something up with my changes to Deno, there's also a benjamn/deno:unmodified image you can use for comparison (no AsyncContext provided).

Please let me know what you think!

[bartlomieju]

FYI a polyfill for AsyncLocalStorage is now available in https://deno.land/std/node/async_hooks.ts. @lucacasonato is working through the comitees on AsyncContext proposal.

[selbyk]

@bartlomieju https://deno.land/std/node/async_hooks.ts 404s after 0.177.0. Any word on this?

[bartlomieju]

You can use it by importing it directly from node:async_hooks module:

import { AsyncLocalStorage } from "node:async_hooks";

[yoshixmk]

@bartlomieju Is it possible to get support for async_hooks.executionAsyncResource() ?

I found in https://github.com/DataDog/dd-trace-js/issues/1892#issuecomment-1560443903

[bartlomieju]

@yoshixmk as it stands we don't plan to add support for that API. Currently we only want to support AsyncLocalStorage.