What’s confusing about modules?

[andrewbranch]

After #51669 is merged, I plan to write documentation for it, and try to update/rewrite a bunch of our existing module-related documentation. While there are a lot of good examples in this issue tracker of specific questions and misconceptions about modules and TypeScript’s module-related options, I thought I’d ask explicitly what questions you have and what aspects of the module landscape or our configuration specifically are the most confusing.

[andrewbranch]

I don’t know how anything in that document can be read as an endorsement. It’s a reference page that has to spell out how features work. I prefixed the section with a big disclaimer about what not to do, but eventually I had to put some examples on the page.

I'm wondering if it was a good idea that we simplified the usage of the feature that at the same time we try to warn people about (did it make sense to drop baseUrl in the context of usage with RequireJS? I assume not)

When I removed the need for baseUrl to use paths in TypeScript 4.1, we were not yet as anti-paths as we have grown to be. However, baseUrl has absolutely no use in bundler code or Node.js, while paths remains useful in some cases. I have literally never seen a project use baseUrl for any reason besides to use paths, and yet it creates a broken way to refer to every file that people had to be careful to avoid. They fundamentally never needed to be coupled, so yes, I still think it was absolutely the right choice to untangle them. I will advocate for deprecating baseUrl in 6.0, while paths will likely live forever, however many caveats we attach to it.

[akwodkiewicz]

You are right, I went too far with endorsement. What I was trying to say is that I thought that the usage of paths for imports convenience was not meant to be officially recognized while the docs show it. But reading your reply I understand that my assumptions were wrong and the de facto usage of paths was embraced by the maintaining team (hence the decoupling of baseUrl from paths).

Thank you, @andrewbranch, for sharing your thoughts on this matter.

I have literally never seen a project use baseUrl for any reason besides to use paths, and yet it creates a broken way to refer to every file that people had to be careful to avoid.

Do you mean some specific issue that can happen when using baseUrl? What is this way you're mentioning if I may ask?

[andrewbranch]

{
  "compilerOptions": {
    "baseUrl": "./src"
  }
}

This means that a file ./src/foo.ts can be imported like this:

import foo from "foo";

from any file in the program, no matter its location. All “bare specifiers” like this are looked up in the baseUrl directory before trying node_modules or whatever the moduleResolution setting implies should happen.

Basically, you’re saying that an AMD loader is going to make an HTTP request with URL fragment foo, and there’s an HTTP server serving the built output of the src directory. But I’m not sure how fully that was even thought through when it was a plausible way you’d ship JS to the browser, because the resolved URL of the request is going to be dependent on the URL of the current page the script is executing from. For example, I just opened up the console and did fetch("foo") and it made a request to https://github.com/microsoft/TypeScript/issues/foo, not https://github.com/foo. If I wanted my imports to work as HTTP requests from any page, it seems like I ought to use a leading /, but baseUrl doesn’t allow that. Maybe the AMD loader accounts for this somehow, in a way that modern ESM imports from the browser wouldn’t. Whatever the reason, it’s completely irrelevant to Node.js and bundlers, and is very poorly applicable to browser-based ESM too.

[akwodkiewicz]

@Andarist , coming back to the "imports" thing -- I can't make them work in my project.

Cannot find module '#lib/something' or its corresponding type declarations. ts(2307)

But I also don't understand how TS is supposed to work here. Could I ask for your help here? Please, take a look at the following example.

---

Having these compiler options in tsconfig.json:

"module": "NodeNext",
"moduleResolution": "NodeNext",
"rootDir": ".",
"outDir": "dist",

and these entries in package.json:

"type": "commonjs",
"imports": {
    "#lib/*": {
        "default": "./dist/src/*.js"
    }
}

I'm trying to import anything from the top-level of the package with"

import { something } from '#lib/something'

but it does not work. So here are the questions that could help me understand the resolution algorithm better (and at the same time fix my project configuration 😅):

1. Where should tsc look for the types?

   "exports" define how the package should work for the users of the package, that's why it's obvious paths to .js files in exports are substituted to .d.ts files in the same directory by default. But "imports" define how the package should work for... maintainers of the package? Both users and maintainers? We cannot tell tsc to resolve the imports using the d.ts files, because they first have to be generated by tsc, and it won't be able to generate them if it cannot resolve the imports, because .d.ts files do not exist yet 😵‍💫

   So I suppose that tsc should look at the source code (.ts) files when resolving the imports. In my case ./src/*.ts

2. In my case, where does tsc look for the types for `"default": "./dist/src/*.js"?

   Does it know it should be ./src/*.ts? Or does it look in ./dist/src/*.ts? Or maybe it looks at some other location?

   Is it possible to print this information somehow on my computer to learn where it tries to resolve #lib/something?

3. Should I add a types conditional import?

   If the answer to 2. suggests that tsc looks at a different place than I want it to look, do I have to tell it to look elsewhere using conditionals (for example "types": "./src/*.ts")?

4. Is my imports entry wrong?

   Am I using the wildcard properly? Should the path end with .js?

5. Does the import in source code refer to some specific file that I don't have in my project?

   Maybe import {something} from '#lib/something' is resolved to some specific file like ./src/something.ts where I expect it to resolve to ./src/something/index.ts

6. Are there any compiler options (or other tsconfig options) that have the effect on the feature?

   Maybe I have some more options set with some particular values that break the resolution? I did not paste all of my tsconfig options here on purpose.

EDIT: 7 [META]: Is there a specific place where I could read all about this without using your help and time and/or referring to comments under GH issues 😅

[Andarist]

@akwodkiewicz i'll try to respond to questions sometime later - but when it comes to the given example, could you wrap it up in a repository that I could clone? that would save me some time

[Andarist]

Btw. you can use --traceResolution to see how TS tries to resolve all of this. You might be able to spot the problem in the output.

[akwodkiewicz]

@Andarist, I was preparing the reproduction repository and was able to reproduce the issue, but I started messing with the tsconfig.json + package.json + import statement and I finally made the imports work. I'll post a bigger comment in a moment (currently writing it), just letting you know that you don't have to go all the way with the explanation.

[akwodkiewicz]

It all boiled down to me not understanding how Node works.

Pointing to folders will not work for subpath imports. Your imports have to point to particular files. Or more exactly, the sum of the import mapping and the thing in your import statement has to result in a particular file. I'll explain in the examples below in a while.

"Folders as modules" seem not to work for subpath imports. It's not mentioned explicitly that it won't work, there's just a following line in the docs:

Using package subpath exports or subpath imports can provide the same containment organization benefits as folders as modules, and work for both require and import.

which suggests that "subpath imports" replace "folders as modules". And the "folders as modules" feature itself is marked as legacy anyway.

So if you have a tree like this:

.
|- dist
|  |- foo
|  |  |- index.js
|  |  
|  |- bar
|  |  |-  index.js
|  |
|  |- baz
|     |-  index.js
|
|- package.json

and you want to import from foo in sibling folders bar or baz, then the subpath import #lib/foo has to point exactly to your ./dist/foo/index.js file, like so:

"imports" {
  "#lib/foo": "./dist/foo/index.js"
}

If you don't want to specify a separate subpath for each directory, you can try using wildcards, like this:

"imports" {
  "#lib/*": "./dist/*/index.js"
}

The above will work even for nested directories, because the asterisk is not a proper glob pattern, it's just string substitution.

Now, if you make the mistake of specifying the path mapping like this:

"imports" {
  "#lib/*": "./dist/*"
}

assuming that Node will resolve the folder as a module, then it won't work for '#lib/foo'. You'll need to import from '#lib/foo/index.js'.

Or if you decide to use:

"imports" {
  "#lib/*": "./dist/*.js"
}

then the correct import statement will be even weirder: '#lib/foo/index'.

———-

When I wrote the import statement including the word “index” in my project, the type acquisition worked out of the box, even without adding any conditional “types” entries to “imports”.

Now the answers to my questions are not that important, I guess, but it still would be useful to know how TS manages to find type information for the subpath imports. I believe it relies on the information from rootDir in tsconfig.json. But if I knew how it works exactly, then I'd know if it's even necessary for me to write a nested "types" entry, and what are the cases that require it.

[andrewbranch]

Yeah, that is a tricky subtlety. Note that it applies to "exports" too.

{
  "name": "foo",
  "exports": {
    "./*": "./dist/*"
  }
}

// main.mjs
import "foo/bar.js";

// main.cjs
require("foo/bar")

Given a file node_modules/foo/dist/bar.js, you might expect each of these to work—the ESM import has to supply the file extension while the CJS require is allowed to drop it as usual. But the normal rules don’t apply once you go through an imports or exports mapping. Like you said, the result of the wildcard substitution has to be a full filename (relative to the package.json). Node.js really wanted the imports/exports algorithm to produce exactly zero or one file lookup location per request for performance reasons (file system access is expensive, to say nothing of HTTP requests in a hypothetical world where these resolutions might take place across a network).

This is also one way the imports/exports resolution algorithm diverges from the tsconfig paths algorithm. With paths, extension searching and directory modules still work on the result of the wildcard substitution, if it would happen for an equivalent request that didn’t map through paths.

[akwodkiewicz]

But the normal rules don’t apply once you go through an imports or exports mapping

I have just said jokingly to a colleague that “Node forgot to backport the folders as modules feature to imports”.

I understand the decision, and seeing the feature marked as legacy helped me tie the pieces together. That was the similarity of exports and imports I failed to recognise.

I thought the similarity is the support of the conditional “types” entry. I got too fixated on trying to point the type sources to TypeScript with the conditional entry, that I just did not realize that I’m writing improper code, resulting in Node runtime errors after transpilation.

This is also one way the imports/exports resolution algorithm diverges from the tsconfig paths algorithm. With paths, extension searching and directory modules still work on the result of the wildcard substitution, if it would happen for an equivalent request that didn’t map through paths.

And the reason it will still “work in runtime” (when translated with the help of bundlers or tools like tsconfig-paths) is that these alias paths end up being relative paths. And “folders as modules” is a feature working exactly just with the relative paths.