microsoft / TypeScript

TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
https://www.typescriptlang.org
Apache License 2.0
101.02k stars 12.49k forks source link

Support for NodeJS 12.7+ package exports #33079

Closed rbuckton closed 2 years ago

rbuckton commented 5 years ago

NodeJS 12.7 added support for a (currently experimental) feature for custom package imports and exports in package.json: https://github.com/jkrems/proposal-pkg-exports/

In short, this feature allows a package author to redirect exports in their package to alternate locations:

{
  "name": "pkg",
  /* [...] */
  "exports": {
    "./foo": "./target.js",
    "./bar/": "./dist/nested/dir/"
  }
}

This is currently only available when --experiemental-exports is passed to NodeJS, however we should continue to track the development of this feature as it progresses.

rbuckton commented 5 years ago

CC: @weswigham, @danielrosenwasser

weswigham commented 5 years ago

Yeah, I know - I don't think we should implement support for it till it's stabilized - things like how . works are still being discussed in the modules group.

rbuckton commented 5 years ago

I agree, this issue exists primarily to serve as a place for us to track the progress of this feature in NodeJS.

richturner commented 5 years ago

Given node is now at v12.11.1 and I believe v12 will enter LTS soon has this functionality stabilised enough to warrant inclusion in typescript module resolution now?

This will prove very helpful when working with yarn simlinked monorepos as the existing types field isn't enough when there are multiple files not in the package root dir.

weswigham commented 5 years ago

We spoke about it in the modules wg on Wednesday - it's going to unflag with es modules as a whole (even the cjs support), so it can be a reliable fallback-allowing mechanism for pre-esm node. It... Should... Unflag during the node 12 lifetime. But that hasn't happened yet, and details are still being fleshed out~

jgoz commented 4 years ago

Looks like this has unflagged in 13.2.0: https://github.com/nodejs/node/blob/v13.2.0/doc/changelogs/CHANGELOG_V13.md#notable-changes

(with support for exports)

jonaskello commented 4 years ago

Node 14 (next LTS) is scheduled for release 2020-04-21 and I'm guessing it will support exports unflagged as node 13 does and at that point I know I will want to use it :-). Is typescript planning to support exports in the same timeframe as the node 14 release?

weswigham commented 4 years ago

Yeah, we've just been waiting for it to stabilize a bit, since it's still experimental, even if it's unflagged (it emits a warning when it's used in resolution), since we don't want to need to support an old experimental version and a final version (and it does still see significant change - there's discussion about removing the main fallback right now).

okdecm commented 4 years ago

Is there a way around this for the time being? Perhaps manually linking up modules via an index.d.ts?

I'm using path based modules such as @okdecm/my-package/Databases/Postgres and thus have no index to specify as my main. The code now works using the exports option, but tooling such as Visual Studio Code still can't resolve the modules for referencing (due to current lack of TypeScript support).

It'd be ideal if there were a way to explicitly set these in the mean time.

sparebytes commented 4 years ago

@okdecm, try setting the baseUrl and paths. If @my/bar depends on @my/foo then you might have a tsconfig like this:

// packages/bar/tsconfig.json
{
  "compilerOptions": {
    "baseUrl": ".",
    "rootDir": "src",
    "outDir": "dist/lib",
    "composite": true,
    "paths": {
      "@my/foo": [
        "../foo/dist/lib/index"
      ],
      "@my/foo/models": [
        "../foo/dist/lib/a/b/c/d/models"
      ],
      "@my/foo/*": [
        "../foo/dist/lib/*"
      ]
    }
  },
  "references": [{ "path": "../foo/tsconfig.json" }]
}

The paths should point to the emitted file. Remember that the paths are relative to the baseUrl so it might make sense to set the baseUrl a parent folder.

jkrems commented 4 years ago

One of the ideas for conditional exports was to allow things like types per exported path.

{
  "name": "pkg",
  /* [...] */
  "exports": {
    "./foo": {
      "types": "./types/foo.d.ts",
      "default": "./target.js"
    },
    "./bar/": {
      "types": "./types/bar.d.ts",
      "default": "./dist/nested/dir/"
    }
  }
}

This would be great to see in a potential TypeScript integration. :)

jarodburchill commented 4 years ago

Any updates on this?

lukeed commented 4 years ago

I'm not sure I understand what the obstacle is? I would think that TS just needs to continue looking at the types/typings field, and load it as is. It should up to the author to declare multiple module definitions within that single d.ts file. Important TS fields/paths should not be scattered throughout the package.json file – too much room for error.

I have examples of this already. I'd expect these to Just Work ™️ :

This would be backwards compatible too, because a definition file that doesn't contain a declare module wrapper already assumes that the definition is the default / applies to the entire package.

lukeed commented 4 years ago

An interesting tidbit is that if you load use a submodule inside a JS file, eg kleur/colors, within VSCode, the submodule's types are picked up and inferred correctly. Writing the same code inside a .ts file short circuits everything to any type.

import * as colors from 'kleur/colors';

colors.r
// (js) has code completions
// (ts) *crickets*
mh-alahdadian commented 4 years ago

this exports field is stable in node api now so it's the time to ts support it any update?

weswigham commented 4 years ago

Sort-of. Basic usecases are pretty stable, but a lot of conditional-related cases are still under discussion. Anyways, we've missed the mark for inclusion in 4.0, so 4.1 would be the earliest you'd see it.

shrinktofit commented 4 years ago

Would TypeScript itself love to have similar options so we can decide the entry modules of a TypeScript project?

// Lib/tsconfig.json
{compilerOptions: {
  exports: [
    "index.ts" // You can only import the `index.ts` from project `Lib`
  ]
}}
// App/tsconfig.json
{references: [
  "path": "<path-to-project-Lib>"
]}
// App/x.ts
import "<path-to-project-Lib>"; // OK
import "<path-to-project-Lib>/hid"; // Type checking error: no such module `<path-to-project-Lib>/hid` 
Bnaya commented 4 years ago

Resolve (and dependents) support is being worked on https://github.com/browserify/resolve/pull/224

KilianKilmister commented 4 years ago

Is there any in-progress work on this and/or are there roadblocks that would need solving?

With node-v14 sceduled to enter Active LTS in under three months, i'd suspect the number of people wanting to have a feature like this to rise at an increasing rate

panva commented 4 years ago

Is there a way around this for the time being?

I'm currently writing a module composed of submodules only, it's TypeScript compiled for browser, node cjs, node esm. Using package.json conditional exports (wildcard syntax from v14.13.0).

Here's my workaround until TS supports the "exports" resolution scheme

Consuming such module I couldn't figure out how to get intellisense working, nor how to even consume the module from typescript since the types wouldn't load when i import submoduleB from 'module/submoduleB'. "typings" or "types" are not working, they point to a file, not a folder, i can't get TS to compile my module structure into a single .d.ts file. So i enabled --moduleResolution to see what's up and just ended up abusing the typesVersions field to get what i need.

Now my module consumers can

And most importantly, I can only bundle the type definition ONCE, even though the modules are available multiple times using different module syntaxes. I don't need to include them next to each of the module flavours.

My published folder structure

module/
├── dist/
│   ├── browser/
│   │   ├── submoduleA/
│   │   │   └── index.js
│   │   └── submoduleB/
│   │       └── index.js
│   ├── node/
│   |    ├── cjs/
│   |    │   ├── submoduleA/
│   |    │   │   └── index.js
│   |    │   └── submoduleB/
│   |    │       └── index.js
│   |    └── esm/
│   |        ├── package.json ({"type": "module"})
│   |        ├── submoduleA/
│   |        │   └── index.js
│   |        └── submoduleB/
│   |            └── index.js
|   └── types/
|       ├── submoduleA/
|       │   └── index.d.ts
|       └── submoduleB/
|           └── index.d.ts
└── package.json

My package.json contents (important ones)

{
  "exports": {
    "./*": {
      "import": "./dist/node/esm/*.js",
      "browser": "./dist/browser/*.js",
      "require": "./dist/node/cjs/*.js"
    }
  },
  "typesVersions": {
    "*": { "*": ["./types/*"] }
  },
  "files": [
    "dist"
  ]
}
Example resolution: (Click to expand) ``` ❯ npx tsc --traceResolution --skipLibCheck --target ES2020 --module ES2020 --moduleResolution node some.ts ======== Resolving module 'josev2/jwe/compact' from '/Users/panva/repo/esm/some.ts'. ======== Explicitly specified module resolution kind: 'NodeJs'. Loading module 'josev2/jwe/compact' from 'node_modules' folder, target file type 'TypeScript'. Found 'package.json' at '/Users/panva/repo/esm/node_modules/josev2/package.json'. 'package.json' has a 'typesVersions' field with version-specific path mappings. 'package.json' has a 'typesVersions' entry '*' that matches compiler version '4.0.3', looking for a pattern to match module name 'jwe/compact'. Module name 'jwe/compact', matched pattern '*'. Trying substitution './types/*', candidate module location: './types/jwe/compact'. File '/Users/panva/repo/esm/node_modules/josev2/types/jwe/compact.d.ts' exist - use it as a name resolution result. Resolving real path for '/Users/panva/repo/esm/node_modules/josev2/types/jwe/compact.d.ts', result '/Users/panva/repo/esm/node_modules/josev2/types/jwe/compact.d.ts'. ======== Module name 'josev2/jwe/compact' was successfully resolved to '/Users/panva/repo/esm/node_modules/josev2/types/jwe/compact.d.ts' with Package ID 'josev2/types/jwe/compact.d.ts@1.0.0'. ======== ```
lcswillems commented 4 years ago

@weswigham Why this feature has not been added to 4.1? Is it possible to add it to 4.2?

AbdelrahmanHafez commented 4 years ago

Node v14 is going LTS in 10 days, this is the only thing currently stopping me from using the new exports so far, would love to see this implemented.

I think we'll see a spike of people wanting to use this soon, any idea when/if we can expect support for it?

xiaoxiangmoe commented 4 years ago

Any updates on this?

trusktr commented 3 years ago

This will be great! I can't wait!! Thanks for all the great work on TS so far TS team.

csvn commented 3 years ago

We just encountered this when using exports on a package. The types did not work for <package-name>/test, so we had to manually add paths so that Typescript could find the correct location for the type definitions.

Looking forward to this issue being solved as well 🙏

lukeed commented 3 years ago

Just a follow up to my previous comment(s) – this approach will always work:

// package.json
{
  "name": "foobar",
  "types": "index.d.ts", // root/main module types only
  "files": [
    "*.d.ts", // root types
    "sub1", // all `foobar/sub1` files
    "sub2", // all `foobar/sub2` files
    "dist" // all `foobar` files
  ],
  "exports": {
    ".": {
      "import": "./dist/index.mjs",
      "require": "./dist/index.js"
    },
    "./sub1": {
      "import": "./sub1/index.mjs",
      "require": "./sub1/index.js"
    },
    "./sub2": {
      "import": "./sub2/index.mjs",
      "require": "./sub2/index.js"
    },
    "./package.json": "./package.json"
  },
  // ...
}

And then the built file structure:

foobar
├── dist
│   ├── index.js
│   └── index.mjs
├── sub1
│   ├── index.js
│   ├── index.mjs
│   └── index.d.ts # `foobar/sub1` types only
├── sub2
│   ├── index.js
│   ├── index.mjs
│   └── index.d.ts # `foobar/sub2` types only
├── index.d.ts     # `foobar` (main) types only
└── package.json

TypeScript gives index.d.ts the same default-resolver behavior that Node.js gives index.js. This means the sub1/index.d.ts and sub2/index.d.ts is important. So when you import { foo } from 'foobar/sub1' the sub1 directory is accessed and the index.d.ts within it is autoloaded.

The name & location of the main module's definitions (/index.d.ts in tree above) does not matter, since the "types" field points to its location.

giltayar commented 3 years ago

@lukeed another workaround similar to yours works even if you don't want to change your directory structure. For example, if sub1 is actually in a src/sub1 directory, and exports has "./sub1": "./src/sub1/index.js", your workaround won't work, as your workaround needs the source directory structure to conform to the directory structure exported.

The workaround is to add a package.json in ./sub1, with two fields: main, and types, that point to /src/sub1/index.js and /src/sub1/index.d.ts respectively (or wherever your decide to put your .d.ts files).

I find that this option is better as it decouples your source code directory structure from the exports path structure.

lukeed commented 3 years ago

It's equally tied to directory structure -- you're just juggling/maintaining more files to make it work.

What I presented will work. If you start changing things, you "void the warranty" so to speak 😅

Shiroh1ge commented 3 years ago

Hey @lukeed, thanks for the awesome solution about module exporting. Currently I'm trying to go with your second approach, but TypeScript refuses to recognize the typing files for some reason. Can you take a look to see if I'm doing anything wrong? I've gone so far to add the index.d.ts file directly in the files, but I still get a Cannot find module 'helpers/modules' or its corresponding type declarations. error

Importing from helpers/lib/modules works properly and also importing from helpers/modules works if I suppress the TypeScript compiler with // @ts-ignore. so it seems that for some reason the typings are not recognized. Thanks for the help!

screenshot

lukeed commented 3 years ago

@Shiroh1ge For this, I think you need to take the approach @giltayar just shared. Unlike my example, you are nesting/aliasing directories, which means that the "helpers/modules" doesn't actually line up with the directory structure.

My approach worked because Node.js allows the helpers/modules import, due to your defined ./modules entry, but then TS goes spelunking through the filesystem looking for a ./helpers/modules((.d)?.ts|/index(.d)?.ts match. Won't find anything since everything exists elsewhere.

Again, the approach @giltayar shared would require that you create a helpers/modules/package.json file with a types field. (I think you might only need main field I'd you're not using a bundler (eg Rollup) and are using TS compiler on its own...not sure)

chyzwar commented 3 years ago

I think as part of this TS would need to support conditional exports

https://nodejs.org/api/packages.html#packages_conditional_exports https://webpack.js.org/guides/package-exports/#conditions

I think there would be a new option in tsconfig like targetConditions : ("node" | "browser" | "development" | "production")[]

hiranya911 commented 3 years ago

Hi @giltayar. I've been trying your workaround on a package that I'm developing, but I can't get it to work. My directory structure:

my-package/
   package.json (contains "main" and "exports")   
   lib/
    index.js
    index.d.ts
    app/
      index.js
      index.d.ts
      package.json (added as per your suggestion)

I can use the package in Node.js with require('my-package/app') but importing it in TS doesn't work:

main.ts:1:25 - error TS2307: Cannot find module 'my-package/app' or its corresponding type declarations.

1 import { getApps } from 'my-package/app';

Any thoughts on what I might be missing?

milesj commented 3 years ago

I've been working on a tool that standardizes the NPM package building process called Packemon (https://packemon.dev).

It helps alleviate the subpath exports problem by using Rollup and creating single build output files (and associated type files using --generateDeclaration=api) based on input entry files. Might be a solution to all the problems listed above.

giltayar commented 3 years ago

@hiranya911 you put the package.json in the app directory under lib when it should be in a top level app directory.

bennypowers commented 3 years ago

One of the ideas for conditional exports was to allow things like types per exported path.

{
  "name": "pkg",
  /* [...] */
  "exports": {
    "./foo": {
      "types": "./types/foo.d.ts",
      "default": "./target.js"
    },
    "./bar/": {
      "types": "./types/bar.d.ts",
      "default": "./dist/nested/dir/"
    }
  }
}

This would be great to see in a potential TypeScript integration. :)

Some tools can consume typescript directly, for example esbuild. Though perhaps out of scope for this discussion, have any of those present given thought to how those files might be mapped?

I've tentatively used an esbuild condition in some of my projects.

hiranya911 commented 3 years ago

Thanks @giltayar. That solution worked for me. For anybody else interested, here's my package layout to work around the issue in question.

my-package/
   package.json (contains "main" and "exports")
   app/
      package.json (contains "main" and "types"; points to ../lib/app directory)     
   lib/
     index.js
     index.d.ts
     app/
        index.js
        index.d.ts
trusktr commented 3 years ago

Yeah, this is getting a more painful now that Webpack 5 obeys exports.

We currently can not use libraries with exports in a TypeScript project with Webpack 5 (without inconvenience of configuring TypeScript paths).

I've been fiddling with paths for a half hour, and no luck. This is too difficult for people that don't use paths.

cecilemuller commented 3 years ago

Currently I publish my TS packages using "main" and "types" pointing to the .ts file, but I'm hoping exports will eventually be a semantically cleaner way to do this (I don't publish TS transpiled down to JS because I can't assume what target people use, and don't want to add bloat in their builds by transpiling to ES5).

jessemyers commented 3 years ago

@hiranya911 Your solution was a huge help for me. Paying it forward, here are a few details that weren't quite explicit.

My project has two sub-packages (foo, bar) -- equivalent to app in the original example -- which show up in the top-level package.json as:

{
    ...
    "types": "dist/index.d.ts",
    "files": [
        "dist",
        "*.d.ts",
        "foo",
        "bar"
    ],
    "exports": {
        ".": "./dist/index.js",
        "./foo": "./dist/path/to/foo.js",
        "./bar": "./dist/path/to/bar.js",
        "./package.json": "./package.json"
    },
    ...
}

Each of the these sub-packages has a foo/package.json and bar/package.json file at the top-level. These look like:

{
    "main": "../dist/path/to/foo.js",
    "types": "../dist/path/to/foo.d.ts"
}

The types mapping here is absolutely essential. I'm not certain that we really need the main mapping since it is a duplicate of the value in the top-level package.json.

nathanforce commented 3 years ago

A few questions that I've encountered related to this:

  1. How do we foresee exports working together with types and typesVersions?
  2. I see some overlap with exports + workspaces (now supported by both yarn and npm) and Typescript Project References. Are they different enough to coexist?
teppeis commented 3 years ago

This is workaround for subpath exports using typesVersions. https://github.com/teppeis/typescript-subpath-exports-workaround

package.json

{
  "main": "dist/index.js",
  "types": "dist-types/index.d.ts",
  "exports": {
    ".": "./dist/index.js",
    "./exported": "./dist/exported.js"
  },
  "typesVersions": {
    "*": {
      "exported": ["dist-types/exported"]
    }
  }
}

This config exports only the package root and ./exported.

// Pass
import "typescript-subpath-exports-workaround"
import "typescript-subpath-exports-workaround/exported"

// Error
import "typescript-subpath-exports-workaround/not-exported"
import "typescript-subpath-exports-workaround/dist/exported"
AbdelrahmanHafez commented 3 years ago

Did anyone manage to get it working for plain JavaScript to make VSCode support Intellisense with exports?

flibustier commented 3 years ago

This is workaround for subpath exports using typesVersions. https://github.com/teppeis/typescript-subpath-exports-workaround

This work very well with the Typescript server used by VSCode, we got full support, but when webpack is compiling it (using ts-loader) we got an import error

Module not found: Error: Can't resolve '@bidule/package/submodule' in 'src/file.ts'
teppeis commented 3 years ago

@flibustier In my test project, it works with webpack (ts-loader). If you need help, you can raise an issue in my repo.

ntucker commented 3 years ago

Would also be great if conditionals could be used to provide types for different versions of typescript. It's annoying not being able to let users take advantage of new typescript powers when publishing libraries just to maintain backwards compatibility.

Edit: Already exists: https://www.typescriptlang.org/docs/handbook/declaration-files/publishing.html#version-selection-with-typesversions

KingDarBoja commented 3 years ago

This is workaround for subpath exports using typesVersions. https://github.com/teppeis/typescript-subpath-exports-workaround

This work very well with the Typescript server used by VSCode, we got full support, but when webpack is compiling it (using ts-loader) we got an import error

Module not found: Error: Can't resolve '@bidule/package/submodule' in 'src/file.ts'

I got same issue while trying to use a package with the typesVersions workaround at some Angular 11 project (which uses webpack 4 under the hood). Any ideas?

yaquawa commented 3 years ago

what's the progress on this? This is a critical issue I think.

okdecm commented 3 years ago

what's the progress on this? This is a critical issue I think.

Looks like its expected to be part of 4.3, set to release in May. https://github.com/microsoft/TypeScript/issues/42762

letmaik commented 3 years ago

I just tried 4.3-beta but couldn't get it working. I tried to find some unit test or pull request for it, but no luck either. Is this really implemented? If so, is there an example somewhere? @DanielRosenwasser

theoludwig commented 3 years ago

I think this is a priority for TypeScript, as the whole Ecosystem will gradually migrate CommonJS modules to ESM. And I think personnally that the outputed code from TypeScript should be as close as the source code, if the target is recent like ES2020.