nodejs / next-10

Repository for discussion on strategic directions for next 10 years of Node.js
Other
472 stars 22 forks source link

Re-evaluating Node.js Experimental Features #285

Open RafaelGSS opened 3 weeks ago

RafaelGSS commented 3 weeks ago

I believe it's time to discuss our policies around experimental features in the project. Features like --experimental-network-import are challenging to assess and maintain, especially when there is no active champion or creator overseeing them.

Proposal

I propose that when the creator or champion of an experimental feature becomes inactive, we should:

  1. Re-evaluate the benefits of the feature.
  2. Decide whether to remove the feature entirely or delegate its maintenance to a team.

This approach will help us set clear expectations, assess and review vulnerabilities, and ensure better maintenance of our experimental features.

Recent Example

We recently dropped the --experimental-policy feature because:

Actionable Steps

To facilitate this discussion, I have written a simple script to list all experimental APIs from nodejs/node/docs/api/**. You can find it here.

Can we evaluate these experimental features either asynchronously or through a dedicated call?

File API Group Header Re-Evaluated
doc/api/async_context.md AsyncLocalStorage.bind(fn) AsyncLocalStorage :white_large_square:
doc/api/async_context.md AsyncLocalStorage.snapshot() AsyncLocalStorage :white_large_square:
doc/api/async_context.md asyncLocalStorage.disable() AsyncLocalStorage :white_large_square:
doc/api/async_context.md asyncLocalStorage.enterWith(store) AsyncLocalStorage :white_large_square:
doc/api/async_context.md asyncLocalStorage.exit(callback[, ...args]) AsyncLocalStorage :white_large_square:
doc/api/async_hooks.md AsyncHooks Async hooks :white_large_square:
doc/api/buffer.md buffer.resolveObjectURL(id) Buffer :white_large_square:
doc/api/child_process.md subprocess[Symbol.dispose]() Child Process :white_large_square:
doc/api/cli.md --allow-addons Permission Model
doc/api/cli.md --allow-child-process Permission Model
doc/api/cli.md --allow-fs-read Permission Model
doc/api/cli.md --allow-fs-write Permission Model
doc/api/cli.md --allow-wasi Permission Model
doc/api/cli.md --allow-worker Permission Model
doc/api/cli.md --build-snapshot Startup Snapshot :white_large_square:
doc/api/cli.md --build-snapshot-config Startup Snapshot :white_large_square:
doc/api/cli.md -C condition, --conditions=condition Conditional Exports :white_large_square:
doc/api/cli.md --cpu-prof :white_large_square:
doc/api/cli.md --cpu-prof-dir :white_large_square:
doc/api/cli.md --cpu-prof-interval :white_large_square:
doc/api/cli.md --cpu-prof-name :white_large_square:
doc/api/cli.md --disable-warning=code-or-type :white_large_square:
doc/api/cli.md --expose-gc :white_large_square:
doc/api/cli.md --env-file=config Node Env :white_large_square:
doc/api/cli.md --experimental-default-type=type Loaders :white_large_square:
doc/api/cli.md --experimental-detect-module Loaders :white_large_square:
doc/api/cli.md --experimental-network-imports Loaders :white_large_square:
doc/api/cli.md --experimental-permission Permission Model
doc/api/cli.md --experimental-require-module Loaders :white_large_square:
doc/api/cli.md --experimental-sea-config SEA :white_large_square:
doc/api/cli.md --experimental-shadow-realm :white_large_square:
doc/api/cli.md --experimental-test-module-mocks Test Runner :white_large_square:
doc/api/cli.md --experimental-test-snapshots Test Runner :white_large_square:
doc/api/cli.md --frozen-intrinsics :white_large_square:
doc/api/cli.md --heap-prof :white_large_square:
doc/api/cli.md --heap-prof-dir :white_large_square:
doc/api/cli.md --heap-prof-interval :white_large_square:
doc/api/cli.md --heap-prof-name :white_large_square:
doc/api/cli.md --heapsnapshot-near-heap-limit=max_count :white_large_square:
doc/api/cli.md --import=module Loaders :white_large_square:
doc/api/cli.md --jitless :white_large_square:
doc/api/cli.md --no-experimental-global-navigator :white_large_square:
doc/api/cli.md --run
doc/api/cli.md --snapshot-blob=path Startup Snapshot :white_large_square:
doc/api/cli.md --test-update-snapshots :white_large_square:
doc/api/cli.md NODE_COMPILE_CACHE=dir :white_large_square:
doc/api/cli.md Source map cache :white_large_square:
doc/api/corepack.md Corepack Corepack :white_large_square:
doc/api/crypto.md crypto.hash(algorithm, data[, outputEncoding]) Crypto :white_large_square:
doc/api/dgram.md socket[Symbol.asyncDispose]() :white_large_square:
doc/api/diagnostics_channel.md diagnostics_channel.tracingChannel(nameOrChannels) Diagnostics :white_large_square:
doc/api/diagnostics_channel.md channel.bindStore(store[, transform]) Diagnostics :white_large_square:
doc/api/diagnostics_channel.md channel.unbindStore(store) Diagnostics :white_large_square:
doc/api/diagnostics_channel.md channel.runStores(context, fn[, thisArg[, ...args]]) Diagnostics :white_large_square:
doc/api/diagnostics_channel.md Class: TracingChannel Diagnostics :white_large_square:
doc/api/diagnostics_channel.md tracingChannel.subscribe(subscribers) Diagnostics :white_large_square:
doc/api/diagnostics_channel.md tracingChannel.unsubscribe(subscribers) Diagnostics :white_large_square:
doc/api/diagnostics_channel.md tracingChannel.traceSync(fn[, context[, thisArg[, ...args]]]) Diagnostics :white_large_square:
doc/api/diagnostics_channel.md tracingChannel.tracePromise(fn[, context[, thisArg[, ...args]]]) Diagnostics :white_large_square:
doc/api/diagnostics_channel.md tracingChannel.traceCallback(fn, position, context, thisArg, ...args) Diagnostics :white_large_square:
doc/api/diagnostics_channel.md Built-in Channels :white_large_square:
doc/api/errors.md ERR_INPUT_TYPE_NOT_ALLOWED :white_large_square:
doc/api/errors.md ERR_REQUIRE_CYCLE_MODULE :white_large_square:
doc/api/errors.md ERR_REQUIRE_ASYNC_MODULE :white_large_square:
doc/api/errors.md ERR_REQUIRE_ESM :white_large_square:
doc/api/errors.md ERR_UNKNOWN_FILE_EXTENSION :white_large_square:
doc/api/errors.md ERR_UNKNOWN_MODULE_FORMAT :white_large_square:
doc/api/errors.md ERR_USE_AFTER_CLOSE :white_large_square:
doc/api/errors.md ERR_NETWORK_IMPORT_BAD_RESPONSE :white_large_square:
doc/api/errors.md ERR_NETWORK_IMPORT_DISALLOWED :white_large_square:
doc/api/esm.md import.meta.dirname :white_large_square:
doc/api/esm.md import.meta.filename :white_large_square:
doc/api/esm.md import.meta.resolve(specifier) :white_large_square:
doc/api/esm.md JSON modules ESM :white_large_square:
doc/api/esm.md Wasm modules ESM :white_large_square:
doc/api/esm.md HTTPS and HTTP imports ESM :white_large_square:
doc/api/events.md events.addAbortListener(signal, listener) Events :white_large_square:
doc/api/fs.md filehandle.readableWebStream([options]) File System :white_large_square:
doc/api/fs.md filehandle[Symbol.asyncDispose]() File System :white_large_square:
doc/api/fs.md fsPromises.glob(pattern[, options]) File System :white_large_square:
doc/api/fs.md fs.glob(pattern[, options], callback) File System :white_large_square:
doc/api/fs.md fs.openAsBlob(path[, options]) File System :white_large_square:
doc/api/fs.md fs.globSync(pattern[, options]) File System :white_large_square:
doc/api/fs.md dirent.parentPath File System :white_large_square:
doc/api/globals.md Class: ByteLengthQueuingStrategy :white_large_square:
doc/api/globals.md Class: CompressionStream :white_large_square:
doc/api/globals.md Class: CountQueuingStrategy :white_large_square:
doc/api/globals.md Crypto Crypto :white_large_square:
doc/api/globals.md CryptoKey Crypto :white_large_square:
doc/api/globals.md CustomEvent :white_large_square:
doc/api/globals.md Class: DecompressionStream :white_large_square:
doc/api/globals.md Navigator :white_large_square:
doc/api/globals.md Class: ReadableByteStreamController :white_large_square:
doc/api/globals.md Class: ReadableStream :white_large_square:
doc/api/globals.md Class: ReadableStreamBYOBReader :white_large_square:
doc/api/globals.md Class: ReadableStreamBYOBRequest :white_large_square:
doc/api/globals.md Class: ReadableStreamDefaultController :white_large_square:
doc/api/globals.md Class: ReadableStreamDefaultReader :white_large_square:
doc/api/globals.md SubtleCrypto :white_large_square:
doc/api/globals.md Class: TextDecoderStream :white_large_square:
doc/api/globals.md Class: TextEncoderStream :white_large_square:
doc/api/globals.md TransformStream TransformStream :white_large_square:
doc/api/globals.md TransformStreamDefaultController TransformStreamDefaultController :white_large_square:
doc/api/globals.md WebSocket globals.md :white_large_square:
doc/api/globals.md WritableStream WritableStream :white_large_square:
doc/api/globals.md WritableStreamDefaultController WritableStreamDefaultController :white_large_square:
doc/api/globals.md WritableStreamDefaultWriter WritableStreamDefaultWriter :white_large_square:
doc/api/http.md server[Symbol.asyncDispose]() http.md :white_large_square:
doc/api/http2.md server[Symbol.asyncDispose]() http2.md :white_large_square:
doc/api/https.md server[Symbol.asyncDispose]() https.md :white_large_square:
doc/api/inspector.md Promises API inspector.md :white_large_square:
doc/api/module.md module.register(specifier[, parentURL][, options]) module.md :white_large_square:
doc/api/module.md Customization Hooks module.md :white_large_square:
doc/api/module.md initialize() module.md :white_large_square:
doc/api/module.md resolve(specifier, context, nextResolve) module.md :white_large_square:
doc/api/module.md load(url, context, nextLoad) module.md :white_large_square:
doc/api/module.md Source map v3 support module.md :white_large_square:
doc/api/n-api.md node_api_nogc_env n-api.md :white_large_square:
doc/api/n-api.md node_api_nogc_finalize n-api.md :white_large_square:
doc/api/n-api.md node_api_create_external_string_latin1 n-api.md :white_large_square:
doc/api/n-api.md node_api_create_external_string_utf16 n-api.md :white_large_square:
doc/api/n-api.md node_api_create_property_key_utf16 n-api.md :white_large_square:
doc/api/n-api.md node_api_post_finalizer n-api.md :white_large_square:
doc/api/net.md server[Symbol.asyncDispose]() net.md :white_large_square:
doc/api/packages.md Determining package manager packages.md :white_large_square:
doc/api/packages.md "packageManager" packages.md :white_large_square:
doc/api/process.md process.constrainedMemory() process.md :white_large_square:
doc/api/process.md process.availableMemory() process.md :white_large_square:
doc/api/process.md process.getActiveResourcesInfo() process.md :white_large_square:
doc/api/process.md process.loadEnvFile(path) process.md :white_large_square:
doc/api/process.md process.setSourceMapsEnabled(val) process.md :white_large_square:
doc/api/process.md process.sourceMapsEnabled process.md :white_large_square:
doc/api/readline.md Promises API readline.md :white_large_square:
doc/api/single-executable-applications.md Single executable applications single-executable-applications.md :white_large_square:
doc/api/stream.md writable.writableAborted stream.md :white_large_square:
doc/api/stream.md readable.readableAborted stream.md :white_large_square:
doc/api/stream.md readable.readableDidRead stream.md :white_large_square:
doc/api/stream.md readable[Symbol.asyncDispose]() stream.md :white_large_square:
doc/api/stream.md readable.compose(stream[, options]) stream.md :white_large_square:
doc/api/stream.md readable.iterator([options]) stream.md :white_large_square:
doc/api/stream.md readable.map(fn[, options]) stream.md :white_large_square:
doc/api/stream.md readable.filter(fn[, options]) stream.md :white_large_square:
doc/api/stream.md readable.forEach(fn[, options]) stream.md :white_large_square:
doc/api/stream.md readable.toArray([options]) stream.md :white_large_square:
doc/api/stream.md readable.some(fn[, options]) stream.md :white_large_square:
doc/api/stream.md readable.find(fn[, options]) stream.md :white_large_square:
doc/api/stream.md readable.every(fn[, options]) stream.md :white_large_square:
doc/api/stream.md readable.flatMap(fn[, options]) stream.md :white_large_square:
doc/api/stream.md readable.drop(limit[, options]) stream.md :white_large_square:
doc/api/stream.md readable.take(limit[, options]) stream.md :white_large_square:
doc/api/stream.md readable.reduce(fn[, initial[, options]]) stream.md :white_large_square:
doc/api/stream.md stream.compose(...streams) stream.md :white_large_square:
doc/api/stream.md stream.Readable.fromWeb(readableStream[, options]) stream.md :white_large_square:
doc/api/stream.md stream.Readable.isDisturbed(stream) stream.md :white_large_square:
doc/api/stream.md stream.isErrored(stream) stream.md :white_large_square:
doc/api/stream.md stream.isReadable(stream) stream.md :white_large_square:
doc/api/stream.md stream.Readable.toWeb(streamReadable[, options]) stream.md :white_large_square:
doc/api/stream.md stream.Writable.fromWeb(writableStream[, options]) stream.md :white_large_square:
doc/api/stream.md stream.Writable.toWeb(streamWritable) stream.md :white_large_square:
doc/api/stream.md stream.Duplex.fromWeb(pair[, options]) stream.md :white_large_square:
doc/api/stream.md stream.Duplex.toWeb(streamDuplex) stream.md :white_large_square:
doc/api/test.md Watch mode test.md :white_large_square:
doc/api/test.md Collecting code coverage test.md :white_large_square:
doc/api/test.md Snapshot testing test.md :white_large_square:
doc/api/test.md snapshot test.md :white_large_square:
doc/api/test.md snapshot.setDefaultSnapshotSerializers(serializers) test.md :white_large_square:
doc/api/test.md snapshot.setResolveSnapshotPath(fn) test.md :white_large_square:
doc/api/test.md MockModuleContext test.md :white_large_square:
doc/api/test.md mock.module(specifier[, options]) test.md :white_large_square:
doc/api/test.md MockTimers test.md :white_large_square:
doc/api/test.md context.assert.snapshot(value[, options]) test.md :white_large_square:
doc/api/test.md context.plan(count) test.md :white_large_square:
doc/api/timers.md immediate[Symbol.dispose]() timers.md :white_large_square:
doc/api/timers.md timeout[Symbol.dispose]() timers.md :white_large_square:
doc/api/timers.md timersPromises.scheduler.wait(delay[, options]) timers.md :white_large_square:
doc/api/timers.md timersPromises.scheduler.yield() timers.md :white_large_square:
doc/api/tracing.md Trace events tracing.md :white_large_square:
doc/api/url.md URL.createObjectURL(blob) url.md :white_large_square:
doc/api/url.md URL.revokeObjectURL(id) url.md :white_large_square:
doc/api/util.md util.MIMEType util.MIMEType :white_large_square:
doc/api/util.md util.parseEnv(content) util.md :white_large_square:
doc/api/util.md util.styleText(format, text) util.md :white_large_square:
doc/api/util.md util.transferableAbortController() util.md :white_large_square:
doc/api/util.md util.transferableAbortSignal(signal) util.md :white_large_square:
doc/api/util.md util.aborted(signal, resource) util.md :white_large_square:
doc/api/v8.md v8.queryObjects(ctor[, options]) v8.md :white_large_square:
doc/api/v8.md v8.setHeapSnapshotNearHeapLimit(limit) v8.md :white_large_square:
doc/api/v8.md Startup Snapshot API v8.md :white_large_square:
doc/api/vm.md vm.Module vm.Module :white_large_square:
doc/api/vm.md vm.SourceTextModule vm.SourceTextModule :white_large_square:
doc/api/vm.md vm.SyntheticModule vm.SyntheticModule :white_large_square:
doc/api/vm.md vm.constants.USE_MAIN_CONTEXT_DEFAULT_LOADER vm.md :white_large_square:
doc/api/vm.md vm.measureMemory([options]) vm.md :white_large_square:
doc/api/wasi.md WebAssembly System Interface (WASI) wasi.md :white_large_square:
doc/api/webcrypto.md Ed25519/Ed448/X25519/X448 key pairs webcrypto.md :white_large_square:
doc/api/worker_threads.md port.hasRef() worker_threads.md :white_large_square:

(Feel free to edit this table)

gireeshpunathil commented 3 weeks ago

Re-evaluate the benefits of the feature. Decide whether to remove the feature entirely or delegate its maintenance to a team.

RafaelGSS commented 3 weeks ago

Absolutely @gireeshpunathil.

BridgeAR commented 3 weeks ago

About having a call: I don't think a general call is useful in this case. The topics are driven by different groups of people e.g., crypto, test runner and AsyncLocalStorage. I think it would be good to try to get feedback by each group about the individual status of these.

GeoffreyBooth commented 3 weeks ago

I think it would be good to try to get feedback by each group about the individual status of these.

I think a separate issue with experimental features by topic/team would be good. Maybe a goal would be that for every feature we're leaving as experimental, we both define its stage and create a tracking issue with a to-do list of what's left before it can go stable. We've had tracking issues for several recent experimental features and I've found them useful, and they're good for communicating to users what to expect.

ronag commented 3 weeks ago

As I've mentioned many times before we have a more general problem with experimental features in terms of that we officially say that experimental features may change or be removed while in practice we don't want to do that due to breaking the ecosystem.

mcollina commented 3 weeks ago

I propose:

  1. mark async_hooks stable.
  2. keep dispose experimental until it's settled in the spec (this is true for all other spec'd things, I think we can make it a rule)
  3. mark Web Streams as stable. There are enough people chipping in that I think it's time. Also it's very weird they are not becuase they are the basis for fetch().
ronag commented 3 weeks ago
  1. mark async_hooks stable.

I agree with this. However, are we good with that they don't work properly with esm? Or do they?

mcollina commented 3 weeks ago

However, are we good with that they don't work properly with esm? Or do they?

I have a good feeling that they work better now that most of the loader is sync. We should do another test for that.

RafaelGSS commented 3 weeks ago

What if we start top-to-bottom?

For AsyncHooks we have the following mention on the docs:

Stability: 1 - Experimental. Please migrate away from this API, if you can. We do not recommend using the createHook, AsyncHook, and executionAsyncResource APIs as they have usability issues, safety risks, and performance implications. Async context tracking use cases are better served by the stable AsyncLocalStorage API. If you have a use case for createHook, AsyncHook, or executionAsyncResource beyond the context tracking need solved by AsyncLocalStorage or diagnostics data currently provided by Diagnostics Channel, please open an issue at https://github.com/nodejs/node/issues describing your use case so we can create a more purpose-focused API.

Having API with such a message, is that ok to move to stable? It seems Experimental will always be the place for it until we migrate things away from async_hooks.

RafaelGSS commented 3 weeks ago

Also, I'll go ahead and mark all flags related to the Permission Model as "Re-Evaluated ✅" and keep its status as experimental as we are still developing and getting more feedback along the way.

GeoffreyBooth commented 3 weeks ago

This thread is going to become unmanageable if we discuss specific APIs here. Can we get a new issue for async hooks?

Qard commented 3 weeks ago

I would doc-deprecate async_hooks and focus on the migration away from it for AsyncLocalStorage. While it is largely API stable, it's most certainly not stable from the perspective of being a thing you can trust to not crash your app. It's simply not a safe API and its use should not be encouraged.

I had asked previously for use cases not solved by ALS so we could build out some more purpose-focused APIs. There's also probably a use for a resource-tracking API which is reduced to only doing that without conflating it with promises and non-consumable resource types like http parser.

I would strongly suggest not stabilizing async_hooks in the state it is in presently. 😬

anonrig commented 3 weeks ago

--run can stay the same. just quite recently, I bumped it to release candidate. in a couple of releases, I plan to make it stable.

joyeecheung commented 3 weeks ago

Maybe we should create a new repo for this? And we PR the evaluation as documents that can be easily looked up by posterity?

(or reuse next-10 repo?)

GeoffreyBooth commented 3 weeks ago

Maybe we should create a new repo for this? And we PR the evaluation as documents that can be easily looked up by posterity?

This is a great idea. The evaluation could also include the todo list for what work remains before a feature can become stable.

RafaelGSS commented 2 weeks ago

@nodejs/next-10 can we transfer it to Next-10 and start creating issues to debate each topic? It should be async, no need to add to the agenda.

marco-ippolito commented 2 weeks ago

@nodejs/next-10 can we transfer it to Next-10 and start creating issues to debate each topic? It should be async, no need to add to the agenda.

SGTM