search

gcanti / io-ts

Runtime type system for IO decoding/encoding
https://gcanti.github.io/io-ts/
MIT License
6.68k stars 331 forks source link

Subpath imports in ESM mode #688

Open stazz opened 1 year ago

stazz commented 1 year ago

🐛 Bug report

Please notice that this problem actually spans a wide range of -ts packages. At least the following are affected

Current Behavior

If using the following subpath import:

import * as errorReport from "io-ts/PathReporter";

And then running using ts-node with --esm flag, or ESM-based bundler (e.g. Vite), while things look OK in IDE, at runtime, there will be error:

xyz/node_modules/ts-node/dist-raw/node-internal-modules-esm-resolve.js:352
     throw new ERR_MODULE_NOT_FOUND(
           ^
 CustomError: Cannot find module '/xyz/node_modules/io-ts/PathReporter' imported from /xyz/my-file.ts

This happens because there is no matching entry for PathReporter subpath in package.json file of io-ts package.

Note that everything works fine when in CJS mode.

Expected behavior

I expect subpath imports of io-ts behave successfully at both compile- as well as runtime, both with CJS modules and ESM modules. Without any additional setup.

Reproducible example

Create package.json with io-ts dependency:

{
  "name":  "io-ts-bug-repro",
  "private": true,
  "version": "1.0.0",
  "dependencies": {
    "fp-ts": "2.13.1",
    "io-ts": "2.2.20"
  },
  "devDependencies": {
    "ts-node": "10.9.1",
    "typescript": "4.9.5"
  }
}

Add basic TS configuration to tsconfig.json:

{
  "compilerOptions": {
    "strict": true,
    "module": "es2022",
    "moduleResolution": "node",
    "lib": [
      "ES2022"
    ],
    "target": "ES2022"
  },
  "ts-node": {
    "esm": true,
    "experimentalSpecifierResolution": "node",
  }
}

Create file src/index.ts with the following code:

import * as errorReport from "io-ts/PathReporter";

const testing = errorReport.success();

and try to execute it:

ts-node src

Suggested solution(s)

It looks like a whole family of -ts packages should be migrated to ESM era. The current setup is extremely weird, lacking "type": "module" entry from package.json files, and duplicating .d.ts files, which also use /lib or /es6 imports interchangeably (sometimes ES6 things including from xyz-ts/lib/abc). This results subpath imports for -ts packages being completely unuseable in ESM.

There are I guess many approaches to solve this. I think it is important to retain CJS functionality still, but also enable everything work for ESM without additional setup. I personally flavor this pattern, which I found to be working for all my packages (note that order of "types", "import", and "require" in the file is meaningful!). I keep .d.ts files in their own dist-ts folder, all the CJS-flavored .js files in their own dist-cjs folder, and finally all ESM-flavored .js files in their own dist-esm folder. I ended up with this setup after long experiments with subpath imports, both outside of module, and within.

{
  "name": "package-usable-by-both-cjs-and-esm",
  "type": "module",
  "main": "./dist-cjs/index.js",
  "module": "./dist-esm/index.js",
  "types": "./dist-ts/index.d.ts",
  "exports": {
    ".": {
      "types": "./dist-ts/index.d.ts",
      "import": "./dist-esm/index.js",
      "require": "./dist-cjs/index.js"
    }
  }
}

If I want to enable subpath export for e.g. my-subpath, I need to first modify the exports in package.json:

{
  "exports": {
    ...,
    "./my-subpath": {
        "types": "./dist-ts/my-subpath.d.ts",
        "import": "./dist-esm/my-subpath.js",
        "require": "./dist-cjs/my-subpath.js"
    }
  }
}

And then create the following stub package file to ./my-subpath/package.json path within distributed NPM package (this is only for CJS support):

{
  "type": "module",
  "main": "../dist-cjs/my-subpath.js",
  "module": "../dist-esm/my-subpath.js",
  "types": "../dist-ts/my-subpath.d.ts"
}

Technically, only main property is needed, but the others are just in case.

This setup allows the same subpath imports always to work, no matter whether they are in TS, CJS, or ESM mode. Furthermore, it adhers to DRY principle, so that it doesn't duplicate .d.ts files.

Additional context

Your environment

Software Version(s)
io-ts 2.2.20
fp-ts 2.13.1
TypeScript 4.9.5
stazz commented 1 year ago

FWIW, I got my own application to work after I did modifications summarized by this gist. The rabbithole went pretty deep.

I hope this will be useful for anyone stumbling upon this until ESM mode starts working properly for -ts packages.

Notice that my usecase might be different from yours and there might be additional setup for those exports fields mentioned in the gist to make things work.

stazz commented 1 year ago

Here are the scripts that I use to produce the file layout described in suggested solution part (entrypoint is build:ci script):

{
  "scripts": {
    "build:ci": "yarn run clear-build-artifacts && yarn run compile-d-ts-files && yarn run tsc --outDir ./dist-esm && yarn run tsc --module CommonJS --outDir ./dist-cjs && yarn run format-output-files && yarn run fix-subpath-exports",
    "clear-build-artifacts": "rm -rf dist dist-ts dist-cjs dist-esm build",
    "compile-d-ts-files": "yarn run tsc --removeComments false --emitDeclarationOnly --declaration --declarationDir ./dist-ts && yarn run copy-d-ts-files && yarn run tsc:plain --project tsconfig.out.json",
    "copy-d-ts-files": "find ./src -mindepth 1 -maxdepth 1 -name '*.d.ts' -exec cp {} ./dist-ts +",
    "fix-subpath-exports": "mkdir <SUBPATH> && cp package.json.<SUBPATH> <SUBPATH>/package.json",
    "format-output-files": "find dist-ts -name '*.ts' -type f -exec sh -c \"echo '/* eslint-disable */\n/* eslint-enable prettier/prettier */'\"' | cat - $1 > $1.tmp && mv $1.tmp $1' -- {} \\; && eslint --no-eslintrc --config '.eslintrc.output.ts.cjs' --fix './dist-ts/**/*.ts' && eslint --no-eslintrc --config '.eslintrc.output.cjs' --fix 'dist-cjs/*js' 'dist-esm/*js'",
    "lint": "eslint ./src --ext .ts,.tsx",
    "tsc": "tsc --project tsconfig.build.json",
    "tsc:plain": "tsc",
  }
}

Notice that when dealing with large number of supported subpath exports, like all -ts packages seem to do, one probably wants fix-subpath-exports script to be more generic. One option would be to read subpaths from file, and then iterate each line with for loop, and create directory + copy/generate the stub package.json file.

Here are the TSConfig files used by script:

// tsconfig.out.json
{
  // TS config file for formatting the resulting .d.ts files (<project name>/dist-ts/**/*.d.ts) that end up in NPM package for typing information.
  "extends": "./tsconfig.json",
  "compilerOptions": {
    "rootDir": "./dist-ts",
    "noEmit": true
  },
  "include": [
    "./dist-ts/**/*"
  ]
}
// tsconfig.build.json
{
  // TS config file to use to compile <project name>/src/**/*.ts files in CI.
  "extends": "./tsconfig.json",
  "compilerOptions": {
    // We don't want dangling // eslint-disable-xyz comments, as that will cause errors during formatting output .[m]js files.
    "removeComments": true
  },
}

And then the normal tsconfig.json file as your usual business. The motivation for these is that we would still have nicely auto-formatted (by ESLint, I personally use the Prettier plugin) .d.ts and .js files, instead of barely readable garbage produced by TS compiler and such.

P.S. No need to take this whole process into use. Just pick the parts which you think suits best to whatever approach you are using to build & publish packages. To be fair, I am happy with anything that makes subpath imports work in ESM mode. :)

stazz commented 1 year ago

@gcanti I took a look at the package.json of this project, and I think I could contribute a PR related to this issue. Would you like me to do that, or are you handling this via some internal ideas/pipelines?

gcanti commented 1 year ago

I'm working on a new iteration of io-ts https://github.com/fp-ts/schema that will be soon included in the @effect org. As soon as the new packages will be stable I will backport what's possible to the fp-ts ecosystem. After that I will consider to release major versions of many packages, io-ts included, which will support esm.

stazz commented 1 year ago

Oh, I see, really interesting to hear about the new iteration, as well as it being included to @effect umbrella! Will check it out for sure. 👍