Memoize module resolution

[oleg-codaio]

In a large codebase, tsconfig-paths starts to introduce performance issues as many modules need to be loaded. In https://github.com/dividab/tsconfig-paths/issues/72#issuecomment-472659666, @BJChen990 suggested introducing a cache to avoid hitting the filesystem multiple times for the same path. For me, this resulted in a ~50% speedup.

[Rush]

We also encountered a lot of issues with performance, especially on Windows. fs.statSync is being called excessively - multiple times for the same file. I also see that every node_module resolution triggers the extension looking in multiple locations in the absolute directory. This seems very wasteful. I added some console.logs

// application loads require('function-bind') which exists in node_modules
Loading function-bind
Exists /code/nexus/portal/function-bind
Exists /code/nexus/portal/function-bind.js
Exists /code/nexus/portal/function-bind.json
Exists /code/nexus/portal/function-bind.node
Exists /code/nexus/portal/function-bind.jsx
Exists /code/nexus/portal/function-bind.ts
Exists /code/nexus/portal/function-bind.tsx
Exists /code/nexus/portal/function-bind/index.js
Exists /code/nexus/portal/function-bind/index.json
Exists /code/nexus/portal/function-bind/index.node
Exists /code/nexus/portal/function-bind/index.jsx
Exists /code/nexus/portal/function-bind/index.ts
Exists /code/nexus/portal/function-bind/index.tsx

[oleg-codaio]

A few other improvement ideas:

• Set addMatchAll to be false by default, otherwise tsconfig-paths will try to resolve even files within node_modules, which should be unnecessary
• By default the list of extensions to search is initialized as https://github.com/dividab/tsconfig-paths/blob/6dc51e43111bea24c847f24d8c13debe305b58bd/src/match-path-sync.ts#L69, where .ts / .tsx extensions are at the end. Instead, set that explicitly to ['.ts', '.tsx', '.js'] (though this may depend on the codebase)
• Optimize https://github.com/dividab/tsconfig-paths/blob/6dc51e43111bea24c847f24d8c13debe305b58bd/src/try-path.ts#L41 to only stat the file by direct name if the requested path actually has an extension (i.e., if (path.extname(physicalPath)) { ... })

[Rush]

Wow, adding addMatchAll: false directly to the installed js files made all the difference. I no longer see so many useless operations to check for file presence. Now I wonder what would be the best way to add this option without breaking the workflow. Maybe an additional option group in tsconfig.json just like Angular does with its angularCompilerOptions ?

[jonaskello]

Have you checked the bootstrapping example in the readme? Calling the tsconfig-paths API in your own wrapper scripts gives you more control.

EDIT: You can see all the params of the register call in the readme.

[Rush]

Honestly I like the simplicity of -r tsconfig-paths/register as this way I can also run various utility scripts in my repo without having a custom wrapper/entrypoint for each.

But I guess I could build my own simple -r ./setup-tsconfig-paths.js

[Rush]

Btw. on Windows the default fileExists checking is absolutely insane. Insanely slow.

There are few reasons: 1) Poor implementation by Node.JS 2) Windows filesystem is slower than Linux 3) aforementioned lack of caching and sub-optimal algorithms in tsconfig-paths

[jonaskello]

If you want a quick solution for filesystem caching you could probably make a bootstrap scripts that calls the lower level creatematchpath. This function accepts a fileExists function that you can pass in. You could pass a function that cache in a way that suits your needs.

[Rush]

@jonaskello I think it's better to figure out a solution for everyone. So far I'm unblocked by hacking the module directly, but let's figure out a proper code fix.

[jonaskello]

Having a bootstrap script is probably a better way to figure out a solution than hacking the installed script. Once you have a bootstrap script that works well you could contribute back that solution either as an example snippet or a PR to the core API. As inspiration for using createMatchPath here is one example of using it to load esm modules.