Proposal: Bundling TS module type definitions

[weswigham]

Relates to #3159, #4068, #2568, this branch, and this tool.

Goals

• Bundle declarations for TS projects to allow a library to be consumed with a single TS file, despite being many modules internally. Because of complicated internal module dependencies which would rather not be exposed to the consumer, this should flatten the exported module types as best as it can. (Ideally, completely.)

  Proposal

When all of --module, --out, and --declarations are specified, the TS compiler should emit a single amalgamated .d.ts (alongside its single output js file). This .d.ts should be flattened compared to a concatenated .d.ts file. It should report collisions caused by scoping issues and import aliasing when flattening declarations into a single declare module. It should respect access modifiers when generating the DTS (only exporting things explicitly exported and types marked as public).

For example, given the following set of sources: tsconfig.json:

{
  "compilerOptions": {
    "module": "commonjs",
    "declarations": true,
    "out": "mylib.js"
  }
}

a.ts:

export * from './b';
export * from './c';

b.ts:

export interface Foo {}

export class Bar {
    constructor() {
        console.log('');
    }

    do(): Foo { throw new Error('Not implemented.'); }
}

c.ts:

export class Baz {}

should create the .d.ts: mylib.d.ts:

declare module "mylib" {
  export interface Foo {}

  export class Bar {
    constructor()
    do(): Foo
  }

  export class Baz {}
}

rather than: mylib.d.ts:

declare module "mylib/a" {
  export * from "mylib/b";
  export * from "mylib/c";
}
declare module "mylib/b" {
  export interface Foo {}

  export class Bar {
    constructor()
    do(): Foo
  }
}
declare module "mylib/c" {
  export class Baz {}
}
declare module "mylib" {
  export * from "mylib/a";
}

and should report a semantic error when the following is done: a.ts:

export * from './b';
export {Bar as Foo} from './b';
export * from './c';

as there will be multiple members named Foo (an interface and a class), since b.ts has exported interface Foo.

We should also have a semantic error when the following is changed from the original: If we change c.ts:

export class Baz {}
export interface Foo {}

it should be an error in a.ts (since it's blanket exporting b and c), and the error should suggest to alias either c.ts's Foo or b.ts's Foo (or both) when reexporting them in a.

Internally, when flattening this aliasing becomes important - we need to track usages of the two original Foo's across the generated .d.ts and rename it to the alias created when it is reexported.

Unfortunately, to maintain ES6 compatability, while we can warn about this behavior with classes (since it's possible that a developer is unaware they're overriding a prior export), we still need to support it (or do we? The spec leads me to believe that attempting to export multiple members with the same name - even via export * - is an early syntax error). So it would be nice to have a compiler flag to mark the same kind of thing with classes (or namespaces) as an error, but also do the following by default:

We can do automatic name collision resolution, but that can result in unpredictable (or convention-based) public member names... but it must be done, I suppose. We could ignore reexported types since it's appropriate to do so in ES6 (following export * declarations can override previously defined members? maybe? system works this way at present - but that may just be system relying on transpiler implementers to maintain ES6 semantics), then we would need to create "shadowed" types at the appropriate level in the .d.ts - types whose original public access are overridden by later exports but whose types are still required to describe public function argument or return types. Naming these "shadowed" types could be difficult, but given that they only exist for type information and not for access information, a common (re)naming convention could be a desirable solution. Something akin to <typename>_n when n is the shadowed type number for that type, and renaming the shadowed type name to something else (<typename>__n and so on so long as the name still exists) if that collides with another exported type. Classes used in this way are rewritten to interfaces in the .d.ts, since a constructor function likely isn't accessible for a shadowed class (at least not at its generated exported type name).

Any feedback? There's a few alternatives to what I've suggested here, which is possibly the most conservative approach in terms of ability to error early but supporting ES6 semantics best. It's possible to silently ignore interface name collisions and rename those automatically as well, but since they're TS constructs and not ES6, I think it's okay to force more discipline in their usage.

Something I've been considering is also rewriting namespaces as interfaces in the generated .d.ts in this way to further flatten/unify the types, but this... might? not strictly be needed. I haven't come up with a strong case for it.

[gpickell]

If the intent is to offer a single "d.ts" for a single-script library, then that is a complicated problem. As a thought, a lot of d.ts files are becoming quite large. Maybe it is not the end of the world to keep the d.ts files laid out as per the default. However, maybe instead introduce a "friend" concept for d.ts files.

Idea shopping...

lib/src/internal.ts:
/// <friend modules="~" />; // Says that the only src relative modules can reference this one.

index.ts:
import ... from "./internal"; // Works because ./ matches ~ (src root)
import ... from "../folder/internal"; // Works because ../ matches ~ (src root)

(so far the ts would only perform syntax validation)

app.ts:
import ... from "lib"; // References index.d.ts so good.
import ... from "lib/internal"; // Error: type checker sees that this module is not a friend.

Advantages:

• Promotes size sanity of d.ts files.
• Keeps d.ts files laid out as per the src.
• Completely compatible with current declaration parsing.
• The friend concept can extend across NPM projects potentially enriching options there.
• There is no need to add any new configuration options.
• Multiple "entry points" are valid with this approach without any real additional concepts.

Cons: Maybe not what people really want.

[kripod]

Hello,

Thanks to everyone for the valuable conversations above. I would like to ask whether any method is known for bundling declaration maps, especially when Microsoft’s API Extractor is in use.

[octogonz]

I would like to ask whether any method is known for bundling declaration maps, especially when Microsoft’s API Extractor is in use.

@kripod This forum is for the TypeScript compiler. To ask an API Extractor specific question, please create an issue for its GitHub project: https://github.com/microsoft/rushstack/issues

You will need to provide more detail explaining how the declaration maps would be used. Thanks!

[timsuchanek]

For anyone wondering, which of the mentioned solutions works. I tried out a few and this one https://github.com/Swatinem/rollup-plugin-dts worked the easiest for me. Just give it 5 min. Even if you're not a rollup user, for this single feature it's worth it. Just use the default config that is provided:

import dts from "rollup-plugin-dts";

const config = [
  // …
  {
    input: "./my-input/index.d.ts",
    output: [{ file: "dist/my-library.d.ts", format: "es" }],
    plugins: [dts()],
  },
];

export default config;

Install the rollup cli:

npm i -g rollup

and execute

rollup -c

We're even using it with the respectExternal option to pull in external types as we're bundling other libraries as well.

[germanz]

For the webpack users, there is also this option via plugin npm-dts-webpack-plugin:

const NpmDtsPlugin = require('npm-dts-webpack-plugin')

module.exports = {
  ......
  plugins: [
    new NpmDtsPlugin({
      logLevel: 'debug'
    })
  ],
  ......
}

[weswigham]

We consider API Extractor and similar tools to be good community solutions to this and don't really see a pressing need for a builtin one, so I'm going to close my own proposal now. 💖

[lgarron]

We consider API Extractor and similar tools to be good community solutions to this and don't really see a pressing need for a builtin one, so I'm going to close my own proposal now. 💖

I know this may not be the right place to ask, but I've had significant trouble finding a .d.ts bundling tool that is reasonably lightweight and preserves semantics for a set of entry points (and is ideally easy to set up). For example:

• @microsoft/api-extractor pulls in 44 dependencies, containing 3321 total files.
• rollup-plugin-dts pulls in 17 dependencies, containing 296 total files.
• tsup pulls in 11 dependencies, containing 922 total files.

I know we're kinda desensitized to numbers like that, but it's quite a lot to audit/trust. For comparison, typescript is only 1 package (containing 182 total files). esbuild installs 2 packages containing 11 total files (for me on macOS, at least).

Also:

• rollup-plugin-dts and tsup each also install rollup, and it feels like an anti-pattern to pull in a whole additional third-party toolchain, given that this kind of functionality is-built in at the compiler level (≈tsc) for other ecosystems.

• None of the tools listed above are able to output declaration maps at this time:

  • https://github.com/microsoft/rushstack/issues/1886
  • https://github.com/Swatinem/rollup-plugin-dts/issues/113
  • https://tsup.egoist.sh/#generate-typescript-declaration-maps--d-ts-map

• https://api-extractor.com/pages/overview/demo_rollup/ states:

  One significant limitation for .d.ts rollups is the assumption that your package has a single entry point. (If that’s not the case, you probably won’t be able to use this feature of API Extractor, although you can still use the API report and documentation generation features.)

• The tools have different opinions about errors and warnings (e.g. due to import resolution and public visibility), and since TypeScript itself doesn't have this feature there is no canonical way to check "will type bundling for my TS code work with most tools?" (an important question to answer if you don't want to get into a situation where you are unable to switch tools).

• It's not possible to use certain other good TypeScript compilers in the ecosystem (which is a respectable decision for the maintainer of any given project, but it is a source of friction for the ecosystem as a whole):

  • esbuild does not plan to support type output.
  • swc intended to support type output, but the issue has been closed by this point: https://github.com/swc-project/swc/issues/657
  • rome may support this some day, but they're in the process of rewriting the entire project from scratch.

To be clear, I appreciate that all these tools exist! I use tsup for a fairly large project right now, and it works at reasonable speed with good output. But I hope I've made a fair case that these "good" solutions have tradeoffs that don't make them ideal for every project. From my perspective, it would still be significantly beneficial to have a canonical .d.ts bundling implementation in tsc itself.

If anyone has advice on any other alternatives, I'd appreciate hearing of them!

[Swatinem]

• rollup-plugin-dts pulls in 17 dependencies, containing 296 total files.

Its only 1 direct dependency, with an additional transitive dependency. @babel/code-frame is marked as an optionalDependency, though it seems like npm sometimes just unconditionally installs it.

Not sure how you get less deps for tsup as it effectively is based on my plugin.

It's not possible to use certain other good TypeScript compilers in the ecosystem

The problem is rather typescript as a language. You don’t need to explicitly declare return types, as the type checker will infer that for you. But this also means that you do need the fully fledged type checker to generate declaration files, which themselves do declare return types.

I doubt there will be a second implementation of the type checker, with the exact same type inference results anytime soon. So unless that happens, you will have to depend on tsc itself.

None of the tools listed above are able to output declaration maps at this time

I don’t think its that hard to implement it in rollup-plugin-dts. Rewriting the preprocessor using magic-string got me halfway there. Now its just the post-processor thats needed. Rollup itself has excellent sourcemap support and probably combine these 3 different sourcemaps without problems. But yes, its not a priority.

---

Well, as the author of one of those tools, I very much would love for this to be an upstream feature so that none of these tools is necessary.

[timocov]

I know this may not be the right place to ask, but I've had significant trouble finding a .d.ts bundling tool that is reasonably lightweight and preserves semantics for a set of entry points (and is ideally easy to set up). For example:

Just fyi dts-bundle-generator pulls 17 dependencies (2 direct: typescript and yargs) ~190 files (excluding typescript package). But it doesn't support source maps as well 🙁

[notaphplover]

We consider API Extractor and similar tools to be good community solutions to this and don't really see a pressing need for a builtin one, so I'm going to close my own proposal now. sparkling_heart

Would you consider looking for use cases before jumping to conclusions? Could we at least know what your considerations are based on?

I think types are not only stuff you take, transform and you forget about it. Types are, at the same time, inputs and outputs of the compiler. I'm sure @microsoft/api-extractor, rollup-plugin-dts, tsup and other similar packages are great tools but I don't think they are, by itself, enough.

Let's supose we have a monorepo with two packages: A and B, located at the packages folder of the project. Let's supose we have a composite typescript project we use to build our source code:

tsconfig.base.json

{
  "$schema": "http://json.schemastore.org/tsconfig",
  "compilerOptions": {
    "composite": true,
    "paths": {
      "@alphabet/a": ["./packages/a"],
      "@alphabet/b": ["./packages/b"],
    },
    ...other common options
  }
}

tsconfig.packages.json

{
  "$schema": "http://json.schemastore.org/tsconfig",
  "files": [],
  "references": [
    {
      "path": "./packages/a/tsconfig.json"
    },
    {
      "path": "./packages/b/tsconfig.json"
    }
  ]
}

packages/a/tsconfig.json

{
  "$schema": "http://json.schemastore.org/tsconfig",
  "extends": "../../tsconfig.base.json",
  "compilerOptions": {
    "outDir": "./lib",
    "rootDir": "./src",
    ...other options
  },
  "include": ["src"]
}

packages/b/tsconfig.json

{
  "$schema": "http://json.schemastore.org/tsconfig",
  "extends": "../../tsconfig.base.json",
  "compilerOptions": {
    "outDir": "./lib",
    "rootDir": "./src",
    ...other options
  },
  "include": ["src"]
}

If b relies on a types, I can't imagine a way of bundling types and at the same time taking advantage of the composite project. If we build our code with something like tsc --build tsconfig.packages.json, there's no room to use any bundler: when tsc tries to build b, it's expecting a types to be there. Types are expected in the types location of the a package, so any types written in a different location are ignored and any types written in that location are overwritten by tsc when it writes it's own typings.

This just an example I came with, I feel I would be fighting the compiler instead of using it. Maybe I'm missing something.

This is just only an example to point that maybe this proposal was discarted too soon, an example to argue maybe community solutions are not good enought since compilation uses types as both inputs and outputs in flows that are not reachable by anything else than tsc.

[lgarron]

Just fyi dts-bundle-generator pulls 17 dependencies (2 direct: typescript and yargs) ~190 files (excluding typescript package). But it doesn't support source maps as well 🙁

Thanks! Unfortunately, I seem to have issues with multiple entry files — I can't output them to a specified output folder, and by default it seems to try to write files in the source code (ignoring "outDir" even when --project tsconfig.json is specified), which fails if there are any .js files in the source tree.

That would be a blocker for our use case. 😔

[Swatinem]

rollup-plugin-dts handles multiple entry files and "composite projects" just fine. Sure, you need to use the rollup way of configuring all that, so outDir and tsconfig will not work.

IMO thinking in "entry points" the way rollup and all the other bundlers work is way more intuitive than the composite projects builtin to TS. But that sure is a matter of preference.

[unional]

IMO thinking in "entry points" the way rollup and all the other bundlers work is way more intuitive than the composite projects builtin to TS. But that sure is a matter of preference.

One issue when not using the composite project is that development experience suffers because the types are not resolved and updated automatically when making changes.

[Rush]

I've spent multiple hours researching this issue 2 years ago. 2 years forward and I need to publish types from a library that's being built from a complex mono repo via Webpack. How to bundle these damn types with Webpack? There are 10 plugins and all are obsolete and neither of them work. Please help Typescript.

[matthew-dean]

Note that rollup-plugin-dts doesn't work with TS path aliases, making it a non-solution. rollup-plugin-ts works, but isn't compatible with TypeScript 4.7-4.9. So basically there are no solutions right now, if someone uses paths aliases. It's not great that if you run TSC to generate types, it will create .d.ts files that do not resolve paths, making the output unusable for distribution. TSC + paths is essentially broken by default.

[matthew-dean]

@weswigham

We consider API Extractor and similar tools to be good community solutions to this and don't really see a pressing need for a builtin one, so I'm going to close my own proposal now. sparkling_heart

There definitely are no good community solutions to this. Have you seen the size of the configuration file for API Extractor, just to bundle .d.ts? Even with that overhead, in many cases it just doesn't work.

[wdanilo]

We consider API Extractor and similar tools to be good community solutions to this and don't really see a pressing need for a builtin one, so I'm going to close my own proposal now. 💖

What do you mean by "good community solution"? After fighting for an hour with it, I gave up. These are just two of MANY errors I got:

[Hotell]

What do you mean by "good community solution"? After fighting for an hour with it, I gave up. These are just two of MANY errors I got:

That's a known issue which has a workaround https://github.com/microsoft/rushstack/issues/2780. But it's indeed concerning that it doesn't follow whole ECMA syntax

[octogonz]

@wdanilo Just curious, what is on line 58 of task.ts?

Also, what do you get with the --diagnostics CLI parameter?

[matthew-dean]

@wdanilo Yeah API Extractor is pretty bad

[jsejcksn]

Ref: TypeScript’s Migration to Modules > Preserving Our API and Bundling Our Declaration Files

The referenced bundler module (latest commit at the time the article was published): https://github.com/microsoft/TypeScript/blob/3f4d16a25ed82c43d06606da3e2001efc82b58eb/scripts/dtsBundler.mjs

[jakebailey]

I cannot express enough how much nobody should use, let alone look at that bundler. Unless your project is TypeScript itself, it almost assuredly will not do the right thing, nor do I even think what it's doing is a good example of how a usable dts bundler should be written.

[joelday]

@jakebailey tell us how you really feel.

[jsejcksn]

@jakebailey 😄 I suppose most won't read that article anyway, so here's a snippet — it seems empathetic to several of the sentiments expressed previously in this issue:

Just like there are many options for bundling JavaScript, there are many options for bundling .d.ts files: api-extractor, rollup-plugin-dts, tsup, dts-bundle-generator, and so on.

These all satisfy the end requirement of "make a single file", however, the additional requirement to produce a final output which declared our API in namespaces similar to our old output meant that we couldn’t use any of them without a lot of modification.

In the end, we opted to roll our own mini-d.ts bundler suited specifically for our needs. This script clocks in at about 400 lines of code, naively walking each entry point’s exports recursively and emitting declarations as-is.

[jakebailey]

If TypeScript itself had a d.ts bundler, I can assure you that the need for dtsBundler.mjs would not have gone away. The TypeScript compiler predates modules and thus its API was declared using namespaces. When we switched to modules, we had to make a conscious effort to construct our code in a very goofy and restrictive way just to make sure that it matched that format, then write custom code to massage the d.ts output into something that looked the same to our external users. If TS had been modules from the start, none of this would have happened; I doubt I would have even written dtsBundler.mjs. We'd probably be shipping unbundled.

Personally, I think a much more reasonable future is going to be one with #47947/#53463 (or similar, in TS or not), which would more explicitly allow bundlers like esbuild to do what they do for JS source but on types; all of the same considerations about hoisting, renaming, and API shape reconstruction are already things that bundlers have to contend with.

[lgarron]

I thought I'd mention another frustration that can't be resolved by using third-party bundlers: it is difficult or impossible to test new TypeScript versions until significantly after they are released.

TypeScript 5.1 implements an important feature for one of our projects. But because we're using a type bundler that only supports up to TypeScript 5.0, I cannot install typescript@v5.1 without causing npm to freak out (and refuse). I could override that, but npm install would still fail for the repo by default.

Despite being careful to maintain a 100% vanilla TypeScript project without bells or whistles, this leaves us stuck waiting a few weeks or months to hope that other projects update their support[^1], before we can fully test that our project works properly with the TypeScript 5.1 feature we need. This is a rather frustrating experience, and would be avoidable if TypeScript had some sort of reference implementation for type bundling that we could use.

[^1]: I could probably contribute to those projects, but I have about half a dozen much more urgent contributions for other projects that I already need to prioritize.

[timocov]

@lgarron I understand your frustration, but it could be applied to any library you use. Or for example linter - it is possible that it might be slightly behind current compiler version, but it doesn't mean that it should be part of the compiler. I feel like this problem needs to be solved by package's maintainers/community support (e.g. the type bundler I maintain has tests running against typescript@next in CI to make sure that everything is alright in the next version and they also are running weekly in case of not having commits to the tool but new compiler releases) rather than merging tools together.

[unional]

IMO it's essentially the difference between two approaches: the NodeJS/JavaScript distributive approach, vs the GoLang/Rust+Cargo/Bun approach.

[lgarron]

@lgarron I understand your frustration, but it could be applied to any library you use. Or for example linter - it is possible that it might be slightly behind current compiler version, but it doesn't mean that it should be part of the compiler.

I understand that this is the case for project maintenance tools in general, but I think this misses my point. When I publish a TypeScript-based library that is easy for everyone in the ecosystem to use, I'm responsible for producing two transformations of the source code:

• The JS
• The types

Those are what it's all about — they constitute the library. Things like linting help with project maintenance, but do not generally affect these outputs for any given input code.

I'm advocating that the reference tsc compiler should be able to produce a publishable library for vanilla TypeScript project, and that type bundling is an important feature of this.

After all, the main point of TypeScript is to provide developer ergonomics through the type system, and unbundled type output is not very ergonomic. For the main project that I maintain (https://github.com/cubing/cubing.js), npx tsc outputs 359 .d.ts files. To understand the source of some types, you'd have to look through half a dozen files that are filled with many types that should not be visible outside the project[^1]. By contrast, tsup outputs 17 well-crafted files that include just what someone using my library needs.

I fact, I think there's a good argument that that ergonomically bundled types are more important than bundled .js for people using a published library: When you look up a symbol from a library that you're using, you are generally looking at its type files — "Go to Definition" has done this in VSCode for as long as I know, whereas "Go to Source Definition" was only recently added. The .d.ts files are essentially a developer interface, in a way that the .js files often are not.

[^1]: Some of which may have names that are deceptively similar to ones that should be visible.

My comment above was meant to point out that it is not quite practical to build and test publishable library files (JS files and type files) until significantly after a TypeScript release. This makes it harder to prepare our code, and to contribute feedback to the TypeScript project about using upcoming language changes in real-world code.

I feel like this problem needs to be solved by package's maintainers/community support

I think this is a reasonable stance in principle. However, the large amount of upvotes and comments in this thread make it clear that the ecosystem has only developed limited solutions after a decade of TypeScript — hence our advocacy for the TypeScript compiler to take the lead and adopt it as a feature.

[llllvvuu]

When you look up a symbol from a library that you're using, you are generally looking at its type files — "Go to Definition" has done this in VSCode for as long as I know, whereas "Go to Source Definition" was only recently added. The .d.ts files are essentially a developer interface, in a way that the .js files often are not.

Unless there is .d.ts.map, which brings up another point: dts-bundle-generator, api-extractor, and rollup-plugin-dts all do not support .d.ts.map. Additionally, api-extractor and dts-bundle-generator do not have chunking, while rollup-plugin-dts is in maintenance mode and author is in this thread suggesting for this to be in tsc.

dts-buddy is an exciting WIP here. I'm not sure if I like the declare module vs having entrypoints be physical files as per "exports" map (not supported in older Node I'm aware).

Currently I think the best DX for a package might be provided by using tsc with "declarationMap": true and no bundling of the .d.ts, but yes bundling of .js. The good: types and go-to-source work, and the non-exposed exports will still be hidden at runtime (and even at compile-time if "moduleResolution": "nodenext"). The bad: declarations for un-exposed exports will still be in the files, so the download size will still not be ideal.

If we build our code with something like tsc --build tsconfig.packages.json, there's no room to use any bundler: when tsc tries to build b, it's expecting a types to be there.

This seems like maybe the strongest counterargument to having .d.ts bundling be in userland. One could try having the bundler work in-place, but that feels iffy.

[imjuni]

I hope this issue will be discussed again. When we do library projects in TypeScript, we need to generate and bundle .d.ts files for reuse. Since tsc doesn't support this process, we have to find another tool, and there are a lot of them, but they all behave a little differently, so it's a pain to sort through them for our purposes.

Since tsc does not apply the path re-map and absolute path(eg. 'src/modules/Button') to the output when generating a .d.ts file, bundling is impossible without the help of other bundling tools. This is making my experience of using TypeScript in library projects a bad one. For example, Svelte has decided to go back to using JavaScript, and my guess is that it's because of the above.

I think what is needed is for tsc to provide this functionality natively, or for dts-gen to generate bundled .d.ts files using tsconfig.json like tsc does.

[ivancuric]

This is still an extremely frustrating situation.

I have a monorepo project for a library that consists of the main entry point package and a web worker. The web worker is its own npm package as it's a different compile target (iife), won't be exposed (internal package), and has a different lib mode ("lib": ["esnext", "webworker"]) compared to the main package.

Most of the types are in the worker package, and it's a peer dependency of the main package, in order to inherit the types.

The problem is that when I bundle the package, I want to expose the types from the web worker as well. However, with tsc I only get stuff like export type {a} from 'unpublished-worker-package.

So then I need to use api-extractor, output the unnecessary declaration files for the worker and hope it will do its job.

[JPPereira123]

Agreed, we've got an npm workspace that is an API client that imports types from another workspace (that is just the API layer that will never be published), so we need to bundle its types as part of the client tsc build.