search

nonara / ts-patch

Augment the TypeScript compiler to support extended functionality
MIT License
768 stars 26 forks source link

Bundlers loaders are not utilising diagnostics produced by transformer plugins via ts-patch addDiagnostic utility #139

Open artem1458 opened 11 months ago

artem1458 commented 11 months ago

When using webpack with ts-loader and ts-patch - diagnostics produced by transformer plugin is not utilised, the result - compilation is not failing even with reported diagnostics with severity ERROR. Same issue appears when using vite with rollup-plugin-typescript2 and ts-patch. This issue appears in both ts-patch modes - Live Compiler and Persistent Patch.

When using tsc or or tspc diagnostics is utilised and failing compilation as expected.

Minimal reproduction repo: https://github.com/artem1458/ts-patch-webpack

Output from build:tsc script (failed as expected):

Custom transformer is working
src/index.ts:1:7 - error TS1: MyCustomTransformationError

1 class Foo {}
        ~~~

Found 1 error in src/index.ts:1

Output from build:webpack script (not failing build):

Custom transformer is working

Custom transformer is working
asset main.js 1.23 KiB [emitted] (name: main)
./src/index.ts 28 bytes [built] [code generated]
webpack 5.89.0 compiled successfully in 725 ms

Output from build:vite script (not failing build):

vite v5.0.5 building for production...
transforming (1) index.html
Custom transformer is working
✓ 3 modules transformed.
dist/index.html                0.40 kB │ gzip: 0.27 kB
dist/assets/index-0d4UnAVa.js  1.57 kB │ gzip: 0.59 kB
✓ built in 632ms
artem1458 commented 11 months ago

@nonara Hey, sorry for bothering, but do you have any thoughts on this one?

DiFuks commented 11 months ago

I confirm. I'll add that there's a similar issue with ForkTSChecker

DiFuks commented 11 months ago

I conducted some research. For example, I tried using the 'transformProgram': true option. The plugin code looked something like this:

export default function (program: Program, host: CompilerHost) {
  const { ...newProgram } = program;

  return Object.assign(program, {
    getSemanticDiagnostics: () => {
      // some code

      return [];
    }
  });
}

And this works with tscp, but still does not work with fork-ts-checker-webpack-plugin. I dug into the source code of fork-ts-checker-webpack-plugin and noticed that it receives an unpatched ts.Program. In this case, the creation happens using the createSemanticDiagnosticsBuilderProgram method. Perhaps, this method is not being patched.

DiFuks commented 11 months ago

Although such code works with ts-loader @artem1458 fyi

nonara commented 11 months ago

AFAIK, all ts program factory functions call createProgram, which is where we hook in. This means it should be covered regardless of using things like incremental builder, etc.

If you're saying the resulting Program instance isn't your replaced one (and you did a persistent patch), it sounds like the plugin you're using is a different instance of typescript.

You could trace out where the plugin code is getting or requiring the typescript module it uses to for program creation. If it's required, you can use require.resolve to find the path. There's also require cache, which could yield some information.

Given what you've shared, my biggest suspicion would be that your plugin is using an unpatched installation of typescript.

A first fast diagnostic route would be to search node modules for all installations of ts, and if you find one that your plugin is may be using, add a throw to the top of the typescript.js file.

You might also alter the plugin code where it creates program and look for typescript.originalCreateProgram. If that's missing, your version should be unpatched – I'm on mobile so I can't confirm, but I believe that's where we copy the old function

DiFuks commented 11 months ago

Although... I managed to make it work with fork-ts-checker (by turning off the 'build' option in the fork-ts-checker settings). However, the plugin behaves incorrectly. getSemanticDiagnostics is called only once, and 'sourceFile' comes as undefined. Perhaps, this is an issue with fork-ts-checker. In any case, the approach with addDiagnostic does not work with any loader and plugin (ts-loader, fork-ts-checker, rollup-plugin-typescript2). And, most likely, the method of overriding 'program.getSemanticDiagnostics' is not the most appropriate way. Although, a similar approach is used in the default TypeScript plugins.

Additionally, I'll add that ts-loader works even without the 'transformProgram': true option, using code like this:

export default function (program: Program, host: CompilerHost, pluginConfig: CliPluginConfig): TransformerPlugin {
  program.getSemanticDiagnostics = (sourceFile, cancellationToken) => {
    // some code

    return [];
  }

  return () => (sourceFile) => sourceFile;
}

However, fork-ts-checker doesn't work at all with this code and behaves incorrectly with the provided one above.

DiFuks commented 11 months ago

I figured out with fork-ts-checker - it indeed does not invoke afterProgramEmitAndDiagnostics or something similar. It directly uses the 'getSemanticDiagnostics' method without parameters. It returns diagnostics for all files (similar to tspc). However, in ts-loader, for example, it provides diagnostics for each file separately. This is something to consider when implementing the plugin

I overrode getSemanticDiagnostics in the transformProgram: true mode, and everything started working. While it may seem like a workaround to me, there is a solution. Most likely, other plugins and loaders for diagnostics use similar approaches.

nonara commented 11 months ago

Sorry guys. Not sure why, but GitHub mobile app has been messing up today. I had some messages duplicate and others were deleted entirely.

It returns diagnostics for all files (similar to tspc). However, in ts-loader, for example, it provides diagnostics for each file separately.

So what we do pre-transform is hook into emitFilesAndReportErrors. In it, we take the composite allDiagnostics array it creates and make it available to our transformers by associating it with the Program instance in WeakMap.

I just recalled that some of these applications have their own way of filtering diagnostics. Looks like ts-loader does indeed have that, so it's not surprising that this is the case. (options -> ignoreDiagnostics)

Something to bear in mind is that the purpose of ts-patch was to hook into tsc or the compiler API's emit process, which it does accomplish.

So, when it comes to certain cases that do their own magic with the TS compiler API, support can't be guaranteed.

However... These are popular libraries, and making sure it's supported does make sense. I already have a plan outline for better diagnostics filtering for the upcoming new major version, but maybe we can find an easy route here.

I overrode getSemanticDiagnostics in the transformProgram: true mode, and everything started working.

@DiFuks Can you give some more details on what exactly you're doing in your workaround? Also, do you know if emitFilesAndReportErrors is ever called? It sounds like maybe ts-loader is manually calling the API to get diagnostics from Program, which would mean our patched EmitResult doesn't matter (?)

DiFuks commented 11 months ago

do you know if emitFilesAndReportErrors is ever called?

no :(

My workaround looks something like this: ```json { "transform": "ts-overrides-plugin/cli", "transformProgram": true, "overrides": [ { "files": ["./src/modern/*.{ts,tsx}"], "compilerOptions": { "strict": true, }, }, ] }, ``` ```ts import type { PluginConfig } from 'ts-patch'; import { TransformerPlugin } from 'ts-patch/plugin-types'; import { globSync } from 'glob'; import { CompilerHost, CompilerOptions, createProgram, Program, } from 'typescript'; import * as path from 'path'; interface Override { files: string[]; compilerOptions: CompilerOptions; } interface CliPluginConfig extends PluginConfig { overrides: Override[]; } export default function (program: Program, host: CompilerHost, pluginConfig: CliPluginConfig): TransformerPlugin { const getSemanticDiagnostics = program.getSemanticDiagnostics.bind(program); const {overrides} = pluginConfig; const defaultCompilerOptions = program.getCompilerOptions(); const rootPath = defaultCompilerOptions.project ? path.dirname(defaultCompilerOptions.project) : process.cwd(); const overridesWithProgram = overrides.map((override) => { const files = globSync(override.files, { cwd: rootPath, absolute: true, }); return { files, program: createProgram(files, { ...defaultCompilerOptions, ...override.compilerOptions, }), } }); program.getSemanticDiagnostics = (sourceFile, cancellationToken) => { const originalDiagnostics = getSemanticDiagnostics(sourceFile, cancellationToken); // for ForkTsCheckerWebpackPlugin and tspc if (!sourceFile) { const overridesDiagnostics = overridesWithProgram.flatMap((override) => { cancellationToken?.throwIfCancellationRequested() return override.files.flatMap((file) => { cancellationToken?.throwIfCancellationRequested() const sourceFile = override.program.getSourceFile(file); if (!sourceFile) { return []; } return override.program.getSemanticDiagnostics(sourceFile); }); }); return [...originalDiagnostics, ...overridesDiagnostics]; } // for ts-loader const override = overridesWithProgram.find((override) => { return override.files.includes(sourceFile.fileName); }); if (override) { return override.program.getSemanticDiagnostics(sourceFile); } return originalDiagnostics; } return () => (sourceFile) => sourceFile; } ```
Although the original code that worked with tsc looked like this: ```json { "transform": "ts-overrides-plugin/cli", "overrides": [ { "files": ["./src/modern/*.{ts,tsx}"], "compilerOptions": { "strict": true, }, }, ] }, ``` ```ts import type { PluginConfig, TransformerExtras } from 'ts-patch'; import { TransformerPlugin } from 'ts-patch/plugin-types'; import { globSync } from 'glob'; import { CompilerOptions, createProgram, Program, } from 'typescript'; import * as path from 'path'; interface Override { files: string[]; compilerOptions: CompilerOptions; } interface CliPluginConfig extends PluginConfig { overrides: Override[]; } export default function (program: Program, pluginConfig: CliPluginConfig, { diagnostics, addDiagnostic, removeDiagnostic, }: TransformerExtras): TransformerPlugin { const {overrides} = pluginConfig; const defaultCompilerOptions = program.getCompilerOptions(); const rootPath = defaultCompilerOptions.project ? path.dirname(defaultCompilerOptions.project) : process.cwd(); const overridesWithProgram = overrides.map((override) => { const files = globSync(override.files, { cwd: rootPath, absolute: true, }); return { files, program: createProgram(files, { ...defaultCompilerOptions, ...override.compilerOptions, }), } }); overridesWithProgram.forEach((override) => { override.files.forEach((file) => { const sourceFile = override.program.getSourceFile(file); if (!sourceFile) { return; } const diagnostics = override.program.getSemanticDiagnostics(sourceFile); diagnostics.forEach((diagnostic) => { addDiagnostic(diagnostic); }); }); }); return () => { return (sourceFile) => sourceFile; }; } ```
If you're interested, here's what the native plugin for the IDE looks like: ```json { "name": "ts-overrides-plugin", "overrides": [ { "files": ["./src/modern/*.{ts,tsx}"], "compilerOptions": { "strict": true, }, }, ] }, ``` ```ts import ts from 'typescript/lib/tsserverlibrary'; import { globSync } from 'glob'; import path from 'path'; interface Override { files: string[]; compilerOptions: ts.CompilerOptions; } interface IdePluginConfig { overrides: Override[]; } export default function init({ typescript }: { typescript: typeof ts}) { function create(info: ts.server.PluginCreateInfo) { const { overrides } = info.config as IdePluginConfig; const defaultCompilerOptions = info.project.getCompilerOptions(); const rootPath = path.dirname(info.project.getProjectName()); const overridesWithProgram = overrides.map((override) => { const files = globSync(override.files, { cwd: rootPath, absolute: true, }); return { files, options: { ...defaultCompilerOptions, ...override.compilerOptions, }, } }); return new Proxy(info.languageService, { get(target, p, receiver): any { if (p === 'getSemanticDiagnostics') { return (fileName: string) => { const override = overridesWithProgram.find((override) => { return override.files.includes(fileName); }); if (override) { const defaultProgram = target.getProgram(); const sourceFile = defaultProgram?.getSourceFile(fileName); const program = typescript.createProgram([fileName], override.options); return program.getSemanticDiagnostics(sourceFile); } return target.getSemanticDiagnostics(fileName); } } return target[p as keyof ts.LanguageService]; } }) } return { create }; } ```
nonara commented 11 months ago

Thanks for sharing, @DiFuks!

Here are a couple notes that can help:

1. Use the provided ts instance rather than import

You should use ts from the extras: ProgramTransformerExtras property of the signature instead of importing typescript. (In the case of a source transformer, you'd want to use its ts provided in the extras arg)

This ensures you're using the same typescript as what was loaded initially.

2. Your Program transformer has an incorrect return

The "Program Transformer" pattern is meant to take an input Program instance and allow you to modify it or recreate it. It should always return a Program.

It looks like your code returns a source file transformer function pattern. In your case, you'd want to just return the original program. No need to recreate it.

One general note. The reason I didn't add diagnostics modification functions for Program transformer is because you can do the work yourself by directly dealing with the Program instance. In your use-case, a Program transformer is definitely the correct route, and from what I can tell, your method of doing it looks reasonable. I wouldn't actually advise using a Source Transformer for your purposes.

Test

Before you make any changes to your code to address the above, would you mind helping with something?

Step 1

In your workaround code, keep everything as it is (for now), but make the following changes:

export default function (program: Program, host: CompilerHost, pluginConfig: CliPluginConfig, { ts }): TransformerPlugin {
  // ... 
  Error.stackTraceLimit = 10000;
  program.getSemanticDiagnostics = function (sourceFile, cancellationToken) {
    let err = new Error(`originalCreateProgram ${ts.originalCreateProgram != null}`);
    process.stderr.write(JSON.stringify(arguments, null, 2) + '\n\n');
    process.stderr.write(err.stack + '\n');
    process.exit(-1);
  }
  // ...
}

What we did was:

  1. Destructured the provided ts instance in the function signature
    • Note: We don't want to swap out typescript for it at this point, though.
  2. Replaced your getSemanticDiagnostics override with one that gives us a long stacktrace + info on the ts instance (helps us determine patch status)

Step 2

  1. Run it with tspc
  2. Run it with your ts-loader setup

Step 3

Please report back the outputs of each run

DiFuks commented 11 months ago

@nonara Thank you for your help!

Here is the report:

tspc ``` {} Error: originalCreateProgram true at program.getSemanticDiagnostics (/Users/difuks/WebstormProjects/ts-overrides-plugin/dist/cli/index.js:200:19) at emitFilesAndReportErrors (evalmachine.:126163:52) at emitFilesAndReportErrorsAndGetExitStatus (evalmachine.:126194:43) at performCompilation (evalmachine.:189723:24) at executeCommandLineWorker (evalmachine.:189527:9) at executeCommandLine (evalmachine.:189610:14) at tsp.execTsc (evalmachine.:190134:23) at Object. (evalmachine.:190138:5) at Object. (/Users/difuks/WebstormProjects/ts-plugin-ignore-strict-example/node_modules/ts-patch/compiler/tsc.js:19:8) at Module._compile (node:internal/modules/cjs/loader:1356:14) at Module._extensions..js (node:internal/modules/cjs/loader:1414:10) at Module.load (node:internal/modules/cjs/loader:1197:32) at Module._load (node:internal/modules/cjs/loader:1013:12) at Module.require (node:internal/modules/cjs/loader:1225:19) at require (node:internal/modules/helpers:177:18) at Object. (/Users/difuks/WebstormProjects/ts-plugin-ignore-strict-example/node_modules/ts-patch/bin/tspc.js:9:5) at Module._compile (node:internal/modules/cjs/loader:1356:14) at Module._extensions..js (node:internal/modules/cjs/loader:1414:10) at Module.load (node:internal/modules/cjs/loader:1197:32) at Module._load (node:internal/modules/cjs/loader:1013:12) at Function.executeUserEntryPoint [as runMain] (node:internal/modules/run_main:128:12) at node:internal/main/run_main_module:28:49 ```
ts-loader ``` ts-loader: Using ts-patch/compiler@5.3.3. This version may or may not be compatible with ts-loader. [webpack-cli] HookWebpackError: Converting circular structure to JSON --> starting at object with constructor 'SourceFileObject' | property 'statements' -> object with constructor 'Array' | index 0 -> object with constructor 'NodeObject' --- property 'parent' closes the circle at makeWebpackError (/Users/difuks/WebstormProjects/ts-plugin-ignore-strict-example/node_modules/webpack/lib/HookWebpackError.js:48:9) at /Users/difuks/WebstormProjects/ts-plugin-ignore-strict-example/node_modules/webpack/lib/Compilation.js:3068:12 at eval (eval at create (/Users/difuks/WebstormProjects/ts-plugin-ignore-strict-example/node_modules/tapable/lib/HookCodeFactory.js:33:10), :40:1) at fn (/Users/difuks/WebstormProjects/ts-plugin-ignore-strict-example/node_modules/webpack/lib/Compilation.js:481:17) at Hook.eval [as callAsync] (eval at create (/Users/difuks/WebstormProjects/ts-plugin-ignore-strict-example/node_modules/tapable/lib/HookCodeFactory.js:33:10), :38:1) at Hook.CALL_ASYNC_DELEGATE [as _callAsync] (/Users/difuks/WebstormProjects/ts-plugin-ignore-strict-example/node_modules/tapable/lib/Hook.js:18:14) at cont (/Users/difuks/WebstormProjects/ts-plugin-ignore-strict-example/node_modules/webpack/lib/Compilation.js:3065:34) at /Users/difuks/WebstormProjects/ts-plugin-ignore-strict-example/node_modules/webpack/lib/Compilation.js:3113:10 at symbolIterator (/Users/difuks/WebstormProjects/ts-plugin-ignore-strict-example/node_modules/neo-async/async.js:3485:9) at done (/Users/difuks/WebstormProjects/ts-plugin-ignore-strict-example/node_modules/neo-async/async.js:3527:9) at /Users/difuks/WebstormProjects/ts-plugin-ignore-strict-example/node_modules/neo-async/async.js:2830:7 at done (/Users/difuks/WebstormProjects/ts-plugin-ignore-strict-example/node_modules/neo-async/async.js:2865:11) at /Users/difuks/WebstormProjects/ts-plugin-ignore-strict-example/node_modules/neo-async/async.js:2818:7 at /Users/difuks/WebstormProjects/ts-plugin-ignore-strict-example/node_modules/webpack/lib/Compilation.js:4666:18 at /Users/difuks/WebstormProjects/ts-plugin-ignore-strict-example/node_modules/webpack/lib/HookWebpackError.js:68:3 at Hook.eval [as callAsync] (eval at create (/Users/difuks/WebstormProjects/ts-plugin-ignore-strict-example/node_modules/tapable/lib/HookCodeFactory.js:33:10), :6:1) at Cache.store (/Users/difuks/WebstormProjects/ts-plugin-ignore-strict-example/node_modules/webpack/lib/Cache.js:107:20) at ItemCacheFacade.store (/Users/difuks/WebstormProjects/ts-plugin-ignore-strict-example/node_modules/webpack/lib/CacheFacade.js:141:15) at /Users/difuks/WebstormProjects/ts-plugin-ignore-strict-example/node_modules/webpack/lib/Compilation.js:4663:25 at /Users/difuks/WebstormProjects/ts-plugin-ignore-strict-example/node_modules/webpack/lib/Cache.js:93:5 at Hook.eval [as callAsync] (eval at create (/Users/difuks/WebstormProjects/ts-plugin-ignore-strict-example/node_modules/tapable/lib/HookCodeFactory.js:33:10), :6:1) at Cache.get (/Users/difuks/WebstormProjects/ts-plugin-ignore-strict-example/node_modules/webpack/lib/Cache.js:75:18) at ItemCacheFacade.get (/Users/difuks/WebstormProjects/ts-plugin-ignore-strict-example/node_modules/webpack/lib/CacheFacade.js:115:15) at /Users/difuks/WebstormProjects/ts-plugin-ignore-strict-example/node_modules/webpack/lib/Compilation.js:4572:22 at arrayEach (/Users/difuks/WebstormProjects/ts-plugin-ignore-strict-example/node_modules/neo-async/async.js:2405:9) at Object.each (/Users/difuks/WebstormProjects/ts-plugin-ignore-strict-example/node_modules/neo-async/async.js:2846:9) at /Users/difuks/WebstormProjects/ts-plugin-ignore-strict-example/node_modules/webpack/lib/Compilation.js:4561:14 at symbolIterator (/Users/difuks/WebstormProjects/ts-plugin-ignore-strict-example/node_modules/neo-async/async.js:3482:9) at timesSync (/Users/difuks/WebstormProjects/ts-plugin-ignore-strict-example/node_modules/neo-async/async.js:2297:7) at Object.eachLimit (/Users/difuks/WebstormProjects/ts-plugin-ignore-strict-example/node_modules/neo-async/async.js:3463:5) at Compilation.createChunkAssets (/Users/difuks/WebstormProjects/ts-plugin-ignore-strict-example/node_modules/webpack/lib/Compilation.js:4538:12) at /Users/difuks/WebstormProjects/ts-plugin-ignore-strict-example/node_modules/webpack/lib/Compilation.js:3108:14 at Compilation._runCodeGenerationJobs (/Users/difuks/WebstormProjects/ts-plugin-ignore-strict-example/node_modules/webpack/lib/Compilation.js:3204:11) at /Users/difuks/WebstormProjects/ts-plugin-ignore-strict-example/node_modules/webpack/lib/Compilation.js:3045:12 at /Users/difuks/WebstormProjects/ts-plugin-ignore-strict-example/node_modules/webpack/lib/Compilation.js:3289:6 at /Users/difuks/WebstormProjects/ts-plugin-ignore-strict-example/node_modules/neo-async/async.js:2818:7 at done (/Users/difuks/WebstormProjects/ts-plugin-ignore-strict-example/node_modules/neo-async/async.js:3522:9) at /Users/difuks/WebstormProjects/ts-plugin-ignore-strict-example/node_modules/webpack/lib/Compilation.js:3252:8 at /Users/difuks/WebstormProjects/ts-plugin-ignore-strict-example/node_modules/webpack/lib/Compilation.js:3361:32 at /Users/difuks/WebstormProjects/ts-plugin-ignore-strict-example/node_modules/webpack/lib/HookWebpackError.js:68:3 at Hook.eval [as callAsync] (eval at create (/Users/difuks/WebstormProjects/ts-plugin-ignore-strict-example/node_modules/tapable/lib/HookCodeFactory.js:33:10), :6:1) at Cache.store (/Users/difuks/WebstormProjects/ts-plugin-ignore-strict-example/node_modules/webpack/lib/Cache.js:107:20) at ItemCacheFacade.store (/Users/difuks/WebstormProjects/ts-plugin-ignore-strict-example/node_modules/webpack/lib/CacheFacade.js:141:15) at /Users/difuks/WebstormProjects/ts-plugin-ignore-strict-example/node_modules/webpack/lib/Compilation.js:3361:11 at /Users/difuks/WebstormProjects/ts-plugin-ignore-strict-example/node_modules/webpack/lib/Cache.js:93:5 at Hook.eval [as callAsync] (eval at create (/Users/difuks/WebstormProjects/ts-plugin-ignore-strict-example/node_modules/tapable/lib/HookCodeFactory.js:33:10), :6:1) at Cache.get (/Users/difuks/WebstormProjects/ts-plugin-ignore-strict-example/node_modules/webpack/lib/Cache.js:75:18) at ItemCacheFacade.get (/Users/difuks/WebstormProjects/ts-plugin-ignore-strict-example/node_modules/webpack/lib/CacheFacade.js:115:15) at Compilation._codeGenerationModule (/Users/difuks/WebstormProjects/ts-plugin-ignore-strict-example/node_modules/webpack/lib/Compilation.js:3331:9) at /Users/difuks/WebstormProjects/ts-plugin-ignore-strict-example/node_modules/webpack/lib/Compilation.js:3238:11 at arrayIterator (/Users/difuks/WebstormProjects/ts-plugin-ignore-strict-example/node_modules/neo-async/async.js:3467:9) at timesSync (/Users/difuks/WebstormProjects/ts-plugin-ignore-strict-example/node_modules/neo-async/async.js:2297:7) at Object.eachLimit (/Users/difuks/WebstormProjects/ts-plugin-ignore-strict-example/node_modules/neo-async/async.js:3463:5) at runIteration (/Users/difuks/WebstormProjects/ts-plugin-ignore-strict-example/node_modules/webpack/lib/Compilation.js:3218:13) at Compilation._runCodeGenerationJobs (/Users/difuks/WebstormProjects/ts-plugin-ignore-strict-example/node_modules/webpack/lib/Compilation.js:3293:3) at Compilation.codeGeneration (/Users/difuks/WebstormProjects/ts-plugin-ignore-strict-example/node_modules/webpack/lib/Compilation.js:3199:8) at /Users/difuks/WebstormProjects/ts-plugin-ignore-strict-example/node_modules/webpack/lib/Compilation.js:3026:11 at eval (eval at create (/Users/difuks/WebstormProjects/ts-plugin-ignore-strict-example/node_modules/tapable/lib/HookCodeFactory.js:33:10), :15:1) at process.processTicksAndRejections (node:internal/process/task_queues:77:11) -- inner error -- TypeError: Converting circular structure to JSON --> starting at object with constructor 'SourceFileObject' | property 'statements' -> object with constructor 'Array' | index 0 -> object with constructor 'NodeObject' --- property 'parent' closes the circle at JSON.stringify () at program.getSemanticDiagnostics (/Users/difuks/WebstormProjects/ts-overrides-plugin/dist/cli/index.js:244:35) at provideErrorsToWebpack (/Users/difuks/WebstormProjects/ts-plugin-ignore-strict-example/node_modules/ts-loader/dist/after-compile.js:138:18) at /Users/difuks/WebstormProjects/ts-plugin-ignore-strict-example/node_modules/ts-loader/dist/after-compile.js:36:9 at /Users/difuks/WebstormProjects/ts-plugin-ignore-strict-example/node_modules/ts-loader/dist/instances.js:206:13 at fn (/Users/difuks/WebstormProjects/ts-plugin-ignore-strict-example/node_modules/webpack/lib/Compilation.js:479:10) at Hook.eval [as callAsync] (eval at create (/Users/difuks/WebstormProjects/ts-plugin-ignore-strict-example/node_modules/tapable/lib/HookCodeFactory.js:33:10), :38:1) at Hook.CALL_ASYNC_DELEGATE [as _callAsync] (/Users/difuks/WebstormProjects/ts-plugin-ignore-strict-example/node_modules/tapable/lib/Hook.js:18:14) at cont (/Users/difuks/WebstormProjects/ts-plugin-ignore-strict-example/node_modules/webpack/lib/Compilation.js:3065:34) at /Users/difuks/WebstormProjects/ts-plugin-ignore-strict-example/node_modules/webpack/lib/Compilation.js:3113:10 at symbolIterator (/Users/difuks/WebstormProjects/ts-plugin-ignore-strict-example/node_modules/neo-async/async.js:3485:9) at done (/Users/difuks/WebstormProjects/ts-plugin-ignore-strict-example/node_modules/neo-async/async.js:3527:9) at /Users/difuks/WebstormProjects/ts-plugin-ignore-strict-example/node_modules/neo-async/async.js:2830:7 at done (/Users/difuks/WebstormProjects/ts-plugin-ignore-strict-example/node_modules/neo-async/async.js:2865:11) at /Users/difuks/WebstormProjects/ts-plugin-ignore-strict-example/node_modules/neo-async/async.js:2818:7 at /Users/difuks/WebstormProjects/ts-plugin-ignore-strict-example/node_modules/webpack/lib/Compilation.js:4666:18 at /Users/difuks/WebstormProjects/ts-plugin-ignore-strict-example/node_modules/webpack/lib/HookWebpackError.js:68:3 at Hook.eval [as callAsync] (eval at create (/Users/difuks/WebstormProjects/ts-plugin-ignore-strict-example/node_modules/tapable/lib/HookCodeFactory.js:33:10), :6:1) at Cache.store (/Users/difuks/WebstormProjects/ts-plugin-ignore-strict-example/node_modules/webpack/lib/Cache.js:107:20) at ItemCacheFacade.store (/Users/difuks/WebstormProjects/ts-plugin-ignore-strict-example/node_modules/webpack/lib/CacheFacade.js:141:15) at /Users/difuks/WebstormProjects/ts-plugin-ignore-strict-example/node_modules/webpack/lib/Compilation.js:4663:25 at /Users/difuks/WebstormProjects/ts-plugin-ignore-strict-example/node_modules/webpack/lib/Cache.js:93:5 at Hook.eval [as callAsync] (eval at create (/Users/difuks/WebstormProjects/ts-plugin-ignore-strict-example/node_modules/tapable/lib/HookCodeFactory.js:33:10), :6:1) at Cache.get (/Users/difuks/WebstormProjects/ts-plugin-ignore-strict-example/node_modules/webpack/lib/Cache.js:75:18) at ItemCacheFacade.get (/Users/difuks/WebstormProjects/ts-plugin-ignore-strict-example/node_modules/webpack/lib/CacheFacade.js:115:15) at /Users/difuks/WebstormProjects/ts-plugin-ignore-strict-example/node_modules/webpack/lib/Compilation.js:4572:22 at arrayEach (/Users/difuks/WebstormProjects/ts-plugin-ignore-strict-example/node_modules/neo-async/async.js:2405:9) at Object.each (/Users/difuks/WebstormProjects/ts-plugin-ignore-strict-example/node_modules/neo-async/async.js:2846:9) at /Users/difuks/WebstormProjects/ts-plugin-ignore-strict-example/node_modules/webpack/lib/Compilation.js:4561:14 at symbolIterator (/Users/difuks/WebstormProjects/ts-plugin-ignore-strict-example/node_modules/neo-async/async.js:3482:9) at timesSync (/Users/difuks/WebstormProjects/ts-plugin-ignore-strict-example/node_modules/neo-async/async.js:2297:7) at Object.eachLimit (/Users/difuks/WebstormProjects/ts-plugin-ignore-strict-example/node_modules/neo-async/async.js:3463:5) at Compilation.createChunkAssets (/Users/difuks/WebstormProjects/ts-plugin-ignore-strict-example/node_modules/webpack/lib/Compilation.js:4538:12) at /Users/difuks/WebstormProjects/ts-plugin-ignore-strict-example/node_modules/webpack/lib/Compilation.js:3108:14 at Compilation._runCodeGenerationJobs (/Users/difuks/WebstormProjects/ts-plugin-ignore-strict-example/node_modules/webpack/lib/Compilation.js:3204:11) at /Users/difuks/WebstormProjects/ts-plugin-ignore-strict-example/node_modules/webpack/lib/Compilation.js:3045:12 at /Users/difuks/WebstormProjects/ts-plugin-ignore-strict-example/node_modules/webpack/lib/Compilation.js:3289:6 at /Users/difuks/WebstormProjects/ts-plugin-ignore-strict-example/node_modules/neo-async/async.js:2818:7 at done (/Users/difuks/WebstormProjects/ts-plugin-ignore-strict-example/node_modules/neo-async/async.js:3522:9) at /Users/difuks/WebstormProjects/ts-plugin-ignore-strict-example/node_modules/webpack/lib/Compilation.js:3252:8 at /Users/difuks/WebstormProjects/ts-plugin-ignore-strict-example/node_modules/webpack/lib/Compilation.js:3361:32 at /Users/difuks/WebstormProjects/ts-plugin-ignore-strict-example/node_modules/webpack/lib/HookWebpackError.js:68:3 at Hook.eval [as callAsync] (eval at create (/Users/difuks/WebstormProjects/ts-plugin-ignore-strict-example/node_modules/tapable/lib/HookCodeFactory.js:33:10), :6:1) at Cache.store (/Users/difuks/WebstormProjects/ts-plugin-ignore-strict-example/node_modules/webpack/lib/Cache.js:107:20) at ItemCacheFacade.store (/Users/difuks/WebstormProjects/ts-plugin-ignore-strict-example/node_modules/webpack/lib/CacheFacade.js:141:15) at /Users/difuks/WebstormProjects/ts-plugin-ignore-strict-example/node_modules/webpack/lib/Compilation.js:3361:11 at /Users/difuks/WebstormProjects/ts-plugin-ignore-strict-example/node_modules/webpack/lib/Cache.js:93:5 at Hook.eval [as callAsync] (eval at create (/Users/difuks/WebstormProjects/ts-plugin-ignore-strict-example/node_modules/tapable/lib/HookCodeFactory.js:33:10), :6:1) at Cache.get (/Users/difuks/WebstormProjects/ts-plugin-ignore-strict-example/node_modules/webpack/lib/Cache.js:75:18) at ItemCacheFacade.get (/Users/difuks/WebstormProjects/ts-plugin-ignore-strict-example/node_modules/webpack/lib/CacheFacade.js:115:15) at Compilation._codeGenerationModule (/Users/difuks/WebstormProjects/ts-plugin-ignore-strict-example/node_modules/webpack/lib/Compilation.js:3331:9) at /Users/difuks/WebstormProjects/ts-plugin-ignore-strict-example/node_modules/webpack/lib/Compilation.js:3238:11 at arrayIterator (/Users/difuks/WebstormProjects/ts-plugin-ignore-strict-example/node_modules/neo-async/async.js:3467:9) at timesSync (/Users/difuks/WebstormProjects/ts-plugin-ignore-strict-example/node_modules/neo-async/async.js:2297:7) at Object.eachLimit (/Users/difuks/WebstormProjects/ts-plugin-ignore-strict-example/node_modules/neo-async/async.js:3463:5) at runIteration (/Users/difuks/WebstormProjects/ts-plugin-ignore-strict-example/node_modules/webpack/lib/Compilation.js:3218:13) at Compilation._runCodeGenerationJobs (/Users/difuks/WebstormProjects/ts-plugin-ignore-strict-example/node_modules/webpack/lib/Compilation.js:3293:3) at Compilation.codeGeneration (/Users/difuks/WebstormProjects/ts-plugin-ignore-strict-example/node_modules/webpack/lib/Compilation.js:3199:8) at /Users/difuks/WebstormProjects/ts-plugin-ignore-strict-example/node_modules/webpack/lib/Compilation.js:3026:11 at eval (eval at create (/Users/difuks/WebstormProjects/ts-plugin-ignore-strict-example/node_modules/tapable/lib/HookCodeFactory.js:33:10), :15:1) at process.processTicksAndRejections (node:internal/process/task_queues:77:11) caused by plugins in Compilation.hooks.processAssets TypeError: Converting circular structure to JSON --> starting at object with constructor 'SourceFileObject' | property 'statements' -> object with constructor 'Array' | index 0 -> object with constructor 'NodeObject' --- property 'parent' closes the circle at JSON.stringify () at program.getSemanticDiagnostics (/Users/difuks/WebstormProjects/ts-overrides-plugin/dist/cli/index.js:244:35) at provideErrorsToWebpack (/Users/difuks/WebstormProjects/ts-plugin-ignore-strict-example/node_modules/ts-loader/dist/after-compile.js:138:18) at /Users/difuks/WebstormProjects/ts-plugin-ignore-strict-example/node_modules/ts-loader/dist/after-compile.js:36:9 at /Users/difuks/WebstormProjects/ts-plugin-ignore-strict-example/node_modules/ts-loader/dist/instances.js:206:13 at fn (/Users/difuks/WebstormProjects/ts-plugin-ignore-strict-example/node_modules/webpack/lib/Compilation.js:479:10) at Hook.eval [as callAsync] (eval at create (/Users/difuks/WebstormProjects/ts-plugin-ignore-strict-example/node_modules/tapable/lib/HookCodeFactory.js:33:10), :38:1) at Hook.CALL_ASYNC_DELEGATE [as _callAsync] (/Users/difuks/WebstormProjects/ts-plugin-ignore-strict-example/node_modules/tapable/lib/Hook.js:18:14) at cont (/Users/difuks/WebstormProjects/ts-plugin-ignore-strict-example/node_modules/webpack/lib/Compilation.js:3065:34) at /Users/difuks/WebstormProjects/ts-plugin-ignore-strict-example/node_modules/webpack/lib/Compilation.js:3113:10 at symbolIterator (/Users/difuks/WebstormProjects/ts-plugin-ignore-strict-example/node_modules/neo-async/async.js:3485:9) at done (/Users/difuks/WebstormProjects/ts-plugin-ignore-strict-example/node_modules/neo-async/async.js:3527:9) at /Users/difuks/WebstormProjects/ts-plugin-ignore-strict-example/node_modules/neo-async/async.js:2830:7 at done (/Users/difuks/WebstormProjects/ts-plugin-ignore-strict-example/node_modules/neo-async/async.js:2865:11) at /Users/difuks/WebstormProjects/ts-plugin-ignore-strict-example/node_modules/neo-async/async.js:2818:7 at /Users/difuks/WebstormProjects/ts-plugin-ignore-strict-example/node_modules/webpack/lib/Compilation.js:4666:18 at /Users/difuks/WebstormProjects/ts-plugin-ignore-strict-example/node_modules/webpack/lib/HookWebpackError.js:68:3 at Hook.eval [as callAsync] (eval at create (/Users/difuks/WebstormProjects/ts-plugin-ignore-strict-example/node_modules/tapable/lib/HookCodeFactory.js:33:10), :6:1) at Cache.store (/Users/difuks/WebstormProjects/ts-plugin-ignore-strict-example/node_modules/webpack/lib/Cache.js:107:20) at ItemCacheFacade.store (/Users/difuks/WebstormProjects/ts-plugin-ignore-strict-example/node_modules/webpack/lib/CacheFacade.js:141:15) at /Users/difuks/WebstormProjects/ts-plugin-ignore-strict-example/node_modules/webpack/lib/Compilation.js:4663:25 at /Users/difuks/WebstormProjects/ts-plugin-ignore-strict-example/node_modules/webpack/lib/Cache.js:93:5 at Hook.eval [as callAsync] (eval at create (/Users/difuks/WebstormProjects/ts-plugin-ignore-strict-example/node_modules/tapable/lib/HookCodeFactory.js:33:10), :6:1) at Cache.get (/Users/difuks/WebstormProjects/ts-plugin-ignore-strict-example/node_modules/webpack/lib/Cache.js:75:18) at ItemCacheFacade.get (/Users/difuks/WebstormProjects/ts-plugin-ignore-strict-example/node_modules/webpack/lib/CacheFacade.js:115:15) at /Users/difuks/WebstormProjects/ts-plugin-ignore-strict-example/node_modules/webpack/lib/Compilation.js:4572:22 at arrayEach (/Users/difuks/WebstormProjects/ts-plugin-ignore-strict-example/node_modules/neo-async/async.js:2405:9) at Object.each (/Users/difuks/WebstormProjects/ts-plugin-ignore-strict-example/node_modules/neo-async/async.js:2846:9) at /Users/difuks/WebstormProjects/ts-plugin-ignore-strict-example/node_modules/webpack/lib/Compilation.js:4561:14 at symbolIterator (/Users/difuks/WebstormProjects/ts-plugin-ignore-strict-example/node_modules/neo-async/async.js:3482:9) at timesSync (/Users/difuks/WebstormProjects/ts-plugin-ignore-strict-example/node_modules/neo-async/async.js:2297:7) at Object.eachLimit (/Users/difuks/WebstormProjects/ts-plugin-ignore-strict-example/node_modules/neo-async/async.js:3463:5) at Compilation.createChunkAssets (/Users/difuks/WebstormProjects/ts-plugin-ignore-strict-example/node_modules/webpack/lib/Compilation.js:4538:12) at /Users/difuks/WebstormProjects/ts-plugin-ignore-strict-example/node_modules/webpack/lib/Compilation.js:3108:14 at Compilation._runCodeGenerationJobs (/Users/difuks/WebstormProjects/ts-plugin-ignore-strict-example/node_modules/webpack/lib/Compilation.js:3204:11) at /Users/difuks/WebstormProjects/ts-plugin-ignore-strict-example/node_modules/webpack/lib/Compilation.js:3045:12 at /Users/difuks/WebstormProjects/ts-plugin-ignore-strict-example/node_modules/webpack/lib/Compilation.js:3289:6 at /Users/difuks/WebstormProjects/ts-plugin-ignore-strict-example/node_modules/neo-async/async.js:2818:7 at done (/Users/difuks/WebstormProjects/ts-plugin-ignore-strict-example/node_modules/neo-async/async.js:3522:9) at /Users/difuks/WebstormProjects/ts-plugin-ignore-strict-example/node_modules/webpack/lib/Compilation.js:3252:8 at /Users/difuks/WebstormProjects/ts-plugin-ignore-strict-example/node_modules/webpack/lib/Compilation.js:3361:32 at /Users/difuks/WebstormProjects/ts-plugin-ignore-strict-example/node_modules/webpack/lib/HookWebpackError.js:68:3 at Hook.eval [as callAsync] (eval at create (/Users/difuks/WebstormProjects/ts-plugin-ignore-strict-example/node_modules/tapable/lib/HookCodeFactory.js:33:10), :6:1) at Cache.store (/Users/difuks/WebstormProjects/ts-plugin-ignore-strict-example/node_modules/webpack/lib/Cache.js:107:20) at ItemCacheFacade.store (/Users/difuks/WebstormProjects/ts-plugin-ignore-strict-example/node_modules/webpack/lib/CacheFacade.js:141:15) at /Users/difuks/WebstormProjects/ts-plugin-ignore-strict-example/node_modules/webpack/lib/Compilation.js:3361:11 at /Users/difuks/WebstormProjects/ts-plugin-ignore-strict-example/node_modules/webpack/lib/Cache.js:93:5 at Hook.eval [as callAsync] (eval at create (/Users/difuks/WebstormProjects/ts-plugin-ignore-strict-example/node_modules/tapable/lib/HookCodeFactory.js:33:10), :6:1) at Cache.get (/Users/difuks/WebstormProjects/ts-plugin-ignore-strict-example/node_modules/webpack/lib/Cache.js:75:18) at ItemCacheFacade.get (/Users/difuks/WebstormProjects/ts-plugin-ignore-strict-example/node_modules/webpack/lib/CacheFacade.js:115:15) at Compilation._codeGenerationModule (/Users/difuks/WebstormProjects/ts-plugin-ignore-strict-example/node_modules/webpack/lib/Compilation.js:3331:9) at /Users/difuks/WebstormProjects/ts-plugin-ignore-strict-example/node_modules/webpack/lib/Compilation.js:3238:11 at arrayIterator (/Users/difuks/WebstormProjects/ts-plugin-ignore-strict-example/node_modules/neo-async/async.js:3467:9) at timesSync (/Users/difuks/WebstormProjects/ts-plugin-ignore-strict-example/node_modules/neo-async/async.js:2297:7) at Object.eachLimit (/Users/difuks/WebstormProjects/ts-plugin-ignore-strict-example/node_modules/neo-async/async.js:3463:5) at runIteration (/Users/difuks/WebstormProjects/ts-plugin-ignore-strict-example/node_modules/webpack/lib/Compilation.js:3218:13) at Compilation._runCodeGenerationJobs (/Users/difuks/WebstormProjects/ts-plugin-ignore-strict-example/node_modules/webpack/lib/Compilation.js:3293:3) at Compilation.codeGeneration (/Users/difuks/WebstormProjects/ts-plugin-ignore-strict-example/node_modules/webpack/lib/Compilation.js:3199:8) at /Users/difuks/WebstormProjects/ts-plugin-ignore-strict-example/node_modules/webpack/lib/Compilation.js:3026:11 at eval (eval at create (/Users/difuks/WebstormProjects/ts-plugin-ignore-strict-example/node_modules/tapable/lib/HookCodeFactory.js:33:10), :15:1) at process.processTicksAndRejections (node:internal/process/task_queues:77:11) ```
fork-ts-checker (build) ``` Error: originalCreateProgram true at program.getSemanticDiagnostics (/Users/difuks/WebstormProjects/ts-overrides-plugin/dist/cli/index.js:200:19) at getDiagnosticsOfProgram (/Users/difuks/WebstormProjects/ts-plugin-ignore-strict-example/node_modules/fork-ts-checker-webpack-plugin/lib/typescript/worker/lib/diagnostics.js:56:48) at useProgram (/Users/difuks/WebstormProjects/ts-plugin-ignore-strict-example/node_modules/fork-ts-checker-webpack-plugin/lib/typescript/worker/lib/program/program.js:27:137) at /Users/difuks/WebstormProjects/ts-plugin-ignore-strict-example/node_modules/fork-ts-checker-webpack-plugin/lib/typescript/worker/get-issues-worker.js:62:34 at Generator.next () at /Users/difuks/WebstormProjects/ts-plugin-ignore-strict-example/node_modules/fork-ts-checker-webpack-plugin/lib/typescript/worker/get-issues-worker.js:8:71 at new Promise () at __awaiter (/Users/difuks/WebstormProjects/ts-plugin-ignore-strict-example/node_modules/fork-ts-checker-webpack-plugin/lib/typescript/worker/get-issues-worker.js:4:12) at getIssuesWorker (/Users/difuks/WebstormProjects/ts-plugin-ignore-strict-example/node_modules/fork-ts-checker-webpack-plugin/lib/typescript/worker/get-issues-worker.js:25:47) at /Users/difuks/WebstormProjects/ts-plugin-ignore-strict-example/node_modules/fork-ts-checker-webpack-plugin/lib/rpc/expose-rpc.js:46:31 at Generator.next () at /Users/difuks/WebstormProjects/ts-plugin-ignore-strict-example/node_modules/fork-ts-checker-webpack-plugin/lib/rpc/expose-rpc.js:8:71 at new Promise () at __awaiter (/Users/difuks/WebstormProjects/ts-plugin-ignore-strict-example/node_modules/fork-ts-checker-webpack-plugin/lib/rpc/expose-rpc.js:4:12) at process.handleMessage (/Users/difuks/WebstormProjects/ts-plugin-ignore-strict-example/node_modules/fork-ts-checker-webpack-plugin/lib/rpc/expose-rpc.js:37:40) at process.emit (node:events:517:28) at emit (node:internal/child_process:944:14) at process.processTicksAndRejections (node:internal/process/task_queues:83:21) RpcExitError: Process 9321 exited with code 255 Issues checking service aborted - probably out of memory. Check the `memoryLimit` option in the ForkTsCheckerWebpackPlugin configuration. If increasing the memory doesn't solve the issue, it's most probably a bug in the TypeScript. ```

Also, I rewrote the plugin based on your advice. Here's the plugin code along with examples. What I found out:

yarn build:tspc # ✅ working
yarn build:ts-loader # ✅ working
yarn build:fork-ts # ✅ working
yarn watch:tspc # ⛔️not working
yarn watch:ts-loader # ✅ working
yarn watch:fork-ts # ⛔️not working

It's evident that the issues are only with tscp and fork-ts-checker in watch mode. In both cases, the plugin function simply isn't being called. Most likely, the problems are related. I studied the source code of fork-ts-checker and found that in watch mode, it creates watchCompilerHost and instantiates a program through baseWatchCompilerHost.createProgram. For some reason, this method returns an unpatched program instance. I could think that the fork-ts-checker developers 'made it complicated,' but the plugin doesn't work with tscp --watch either

I tried to come up with a drastic solution - override the createWatchCompilerHost method in the plugin code, but unsurprisingly, I got the error message: TypeError: Cannot redefine property: createWatchCompilerHost

I would really like to bring the implementation of my plugin to completion. Since I believe it would be helpful to many people, I'm willing to assist you in fixing it (if you find it necessary)

DiFuks commented 10 months ago

I've made progress in finding a solution. The issue in my case is that getSemanticDiagnostics is called not from the program but from builderProgram. In turn, it calls getSemanticDiagnosticsOfFile. Only then does it call getProgramDiagnostics from the program. getProgramDiagnostics is a private method of the program. I can override it for my needs, but perhaps it will be a somewhat suboptimal workaround

DiFuks commented 10 months ago

The final version of the solution

For some reason, overriding getProgramDiagnostics also didn't yield results - it returned an empty array. In the end, I had to override getBindAndCheckDiagnostics. As a result, everything worked in ts-loader, tspc, forktschecker, both in build and watch modes.

artem1458 commented 7 months ago

@nonara @DiFuks Hey, I finally managed to make fix that should fix this problem https://github.com/nonara/ts-patch/pull/150

@DiFuks I couldn't make it works with fork-ts-checker-webpack-plugin, even with patching getBindAndCheckDiagnostics method, but it works with ts-loader which is good I think

DiFuks commented 7 months ago

@artem1458 Thank you! Oddly, the solution presented here works well with ForkTSChecker.

Unfortunately, my team specifically uses ForkTSChecker. :(

artem1458 commented 6 months ago

@nonara @DiFuks You know, I actually thinking about adding methods that will add different kind of diagnostics to the program

Program have a few methods for providing diagnostics image So we can add methods to the TransformerExtras:

Compiler plugin developers can decide which diagnostics they want to add. @nonara What do you think about it, is it ok to add such methods? I can add them to fix it if you think that it make sense.