CodinGame / monaco-vscode-api

VSCode public API plugged on the monaco editor
MIT License
236 stars 31 forks source link

Treemending failing #113

Closed MrMarble closed 1 year ago

MrMarble commented 1 year ago

Trying to setup monaco-vscode-api with pnpm fails to "treemend"

Here are the logs of a clean (removed node_modules folder) install:

$pnpm i

Lockfile is up to date, resolution step is skipped
Packages: +1246
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Packages are hard linked from the content-addressable store to the virtual store.
  Content-addressable store is at: F:\.pnpm-store\v3
  Virtual store is at:             node_modules/.pnpm
Downloading registry.npmjs.org/monaco-editor/0.37.1: 16 MB/16 MB, done
Progress: resolved 1246, reused 1243, downloaded 2, added 1246, done

dependencies:
+ @monaco-editor/react 4.5.1
+ @tauri-apps/api 1.3.0
+ monaco-editor 0.37.1
+ react 18.2.0
+ react-dom 18.2.0
+ react-resizable-panels 0.0.42
+ react-use 17.4.0
+ tauri-plugin-store-api 0.0.0
+ uuid 9.0.0
+ vscode <- @codingame/monaco-vscode-api 1.78.5
+ zustand 4.3.8

devDependencies:
+ @storybook/addon-essentials 7.0.10
+ @storybook/addon-interactions 7.0.10
+ @storybook/addon-links 7.0.10
+ @storybook/addon-styling 1.0.6
+ @storybook/blocks 7.0.10
+ @storybook/react 7.0.10
+ @storybook/react-vite 7.0.10
+ @storybook/testing-library 0.0.14-next.2
+ @types/mocha 10.0.1
+ @types/node 20.1.0
+ @types/react 18.2.6
+ @types/react-dom 18.2.4
+ @types/uuid 9.0.1
+ @types/vscode 1.78.0
+ @typescript-eslint/eslint-plugin 5.59.2
+ @typescript-eslint/parser 5.59.2
+ @vitejs/plugin-react 4.0.0
+ @wdio/cli 8.10.0
+ @wdio/local-runner 8.10.0
+ @wdio/mocha-framework 8.10.0
+ @wdio/spec-reporter 8.10.0
+ autoprefixer 10.4.14
+ eslint 8.40.0
+ eslint-config-prettier 8.8.0
+ eslint-plugin-react-hooks 4.6.0
+ eslint-plugin-react-refresh 0.4.1
+ eslint-plugin-simple-import-sort 10.0.0
+ vite 4.3.5

> @0.0.1 postinstall F:\repos\runts
> monaco-treemending

Error: Unable to apply patch on F:\repos\runts\node_modules\.pnpm\monaco-editor@0.37.1\node_modules\monaco-editor\esm\vs\base\common\paging.js
    at Object.patched (file:///F:/repos/runts/node_modules/.pnpm/@codingame+monaco-vscode-api@1.78.5_monaco-editor@0.37.1_vscode-oniguruma@1.7.0_vscode-textmate@9.0.0_yauzl@2.10.0/node_modules/@codingame/monaco-vscode-api/dist/monaco-treemending.js:854:30)
    at file:///F:/repos/runts/node_modules/.pnpm/@codingame+monaco-vscode-api@1.78.5_monaco-editor@0.37.1_vscode-oniguruma@1.7.0_vscode-textmate@9.0.0_yauzl@2.10.0/node_modules/@codingame/monaco-vscode-api/dist/monaco-treemending.js:820:15
    at Object.loadFile (file:///F:/repos/runts/node_modules/.pnpm/@codingame+monaco-vscode-api@1.78.5_monaco-editor@0.37.1_vscode-oniguruma@1.7.0_vscode-textmate@9.0.0_yauzl@2.10.0/node_modules/@codingame/monaco-vscode-api/dist/monaco-treemending.js:845:21)

The integrity of 1537 files was checked. This might have caused installation to take longer.
Done in 10.3s
CGNonofr commented 1 year ago

Are you able to provide a minimal repo reproducing the issue? Is it related to pnpm?

MrMarble commented 1 year ago

Are you able to provide a minimal repo reproducing the issue? Is it related to pnpm?

Tried using yarn instead of pnpm, (removed node_modules folder and did yarn install) and seems to work, here's part of the log (yarn reuses the screen):

[4/4] Building fresh packages...
success Saved lockfile.
$ monaco-treemending
Monaco-editor was tree-mended
Done in 67.53s.

So it must be the way pnpm handles packages

kaisalmen commented 1 year ago

@MrMarble doesn't pnpm store the packages centrally and only links them locally? Could explain it. If you ever patched it before you will see the error (that is at least the current behavior with npm)

MrMarble commented 1 year ago

@MrMarble doesn't pnpm store the packages centrally and only links them locally? Could explain it. If you ever patched it before you will see the error (that is at least the current behavior with npm)

will try again clearing the pnpm store so it's like the first install, give me a minute


Same result @kaisalmen :

Lockfile is up to date, resolution step is skipped
Packages: +1246
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++  
Packages are hard linked from the content-addressable store to the virtual store.
  Content-addressable store is at: F:\.pnpm-store\v3
  Virtual store is at:             node_modules/.pnpm
Downloading registry.npmjs.org/typescript/5.0.4: 7.05 MB/7.05 MB, done
Downloading registry.npmjs.org/monaco-editor/0.37.1: 16 MB/16 MB, done
Progress: resolved 1246, reused 0, downloaded 1245, added 1246, done
node_modules/.pnpm/esbuild@0.17.18/node_modules/esbuild: Running postinstall script, done in 402ms

dependencies:
+ @monaco-editor/react 4.5.1
+ @tauri-apps/api 1.3.0
+ monaco-editor 0.37.1
+ react 18.2.0
+ react-dom 18.2.0
+ react-resizable-panels 0.0.42
+ react-use 17.4.0
+ tauri-plugin-store-api 0.0.0
+ uuid 9.0.0
+ vscode <- @codingame/monaco-vscode-api 1.78.5
+ zustand 4.3.8

devDependencies:
+ @storybook/addon-essentials 7.0.10
+ @storybook/addon-interactions 7.0.10
+ @storybook/addon-links 7.0.10
+ @storybook/addon-styling 1.0.6
+ @storybook/blocks 7.0.10
+ @storybook/react 7.0.10
+ @storybook/react-vite 7.0.10
+ @storybook/testing-library 0.0.14-next.2
+ @types/mocha 10.0.1
+ @types/node 20.1.0
+ @types/react 18.2.6
+ @types/react-dom 18.2.4
+ @types/uuid 9.0.1
+ @types/vscode 1.78.0
+ @typescript-eslint/eslint-plugin 5.59.2
+ @typescript-eslint/parser 5.59.2
+ @vitejs/plugin-react 4.0.0
+ @wdio/cli 8.10.0
+ @wdio/local-runner 8.10.0
+ @wdio/mocha-framework 8.10.0
+ @wdio/spec-reporter 8.10.0
+ autoprefixer 10.4.14
+ eslint 8.40.0
+ eslint-config-prettier 8.8.0
+ eslint-plugin-react-hooks 4.6.0
+ eslint-plugin-react-refresh 0.4.1
+ eslint-plugin-simple-import-sort 10.0.0
+ eslint-plugin-storybook 0.6.12
+ eslint-plugin-wdio 8.8.7
+ postcss 8.4.23
+ prettier 2.8.8
+ prettier-plugin-tailwindcss 0.2.8
+ prop-types 15.8.1
+ storybook 7.0.10
+ tailwind-merge 1.12.0
+ tailwindcss 3.3.2
+ typescript 5.0.4
+ vite 4.3.5

> @0.0.1 postinstall F:\repos\runts
> monaco-treemending

Error: Unable to apply patch on F:\repos\runts\node_modules\.pnpm\monaco-editor@0.37.1\node_modules\monaco-editor\esm\vs\base\common\paging.js
    at Object.patched (file:///F:/repos/runts/node_modules/.pnpm/@codingame+monaco-vscode-api@1.78.5_monaco-editor@0.37.1_vscode-oniguruma@1.7.0_vscode-textmate@9.0.0_yauzl@2.10.0/node_modules/@codingame/monaco-vscode-api/dist/monaco-treemending.js:854:30)
    at file:///F:/repos/runts/node_modules/.pnpm/@codingame+monaco-vscode-api@1.78.5_monaco-editor@0.37.1_vscode-oniguruma@1.7.0_vscode-textmate@9.0.0_yauzl@2.10.0/node_modules/@codingame/monaco-vscode-api/dist/monaco-treemending.js:820:15
    at Object.loadFile (file:///F:/repos/runts/node_modules/.pnpm/@codingame+monaco-vscode-api@1.78.5_monaco-editor@0.37.1_vscode-oniguruma@1.7.0_vscode-textmate@9.0.0_yauzl@2.10.0/node_modules/@codingame/monaco-vscode-api/dist/monaco-treemending.js:845:21)

Done in 30.7s
kaisalmen commented 1 year ago

@MrMarble thank you. I will try to reproduce this in the upcoming days and think about how we can make the patch process detect that monaco-editor has already been patched. This problem also came up with monaco-langugeclient.

MrMarble commented 1 year ago

I've been trying to use the previous version 5.0.1 but I can't get it to work. I've installed the versions listed on the README, using monaco-editor/react and added the imports but I get an error:

X [ERROR] Could not resolve "env"

    vite-plugin-wasm-namespace:F:\repos\runts\node_modules\.pnpm\vscode-oniguruma@1.7.0\node_modules\vscode-oniguruma\release\onig.wasm:43:160:
      43 │ ...t { emscripten_memcpy_big as __vite__wasmImport_0_0, emscripten_get_now as __vite__wasmImport_0_1, emscripten_resize_heap as __vite__wasmImport_0_2 } from "env"; 
         ╵                                                                                                                                                               ~~~~~  

  You can mark the path "env" as external to exclude it from the bundle, which will remove this
  error.

X [ERROR] Could not resolve "wasi_snapshot_preview1"

    vite-plugin-wasm-namespace:F:\repos\runts\node_modules\.pnpm\vscode-oniguruma@1.7.0\node_modules\vscode-oniguruma\release\onig.wasm:44:51:
      44 │ import { fd_write as __vite__wasmImport_1_0 } from "wasi_snapshot_preview1";
         ╵                                                    ~~~~~~~~~~~~~~~~~~~~~~~~

  You can mark the path "wasi_snapshot_preview1" as external to exclude it from the bundle, which
  will remove this error.

F:\repos\runts\node_modules\.pnpm\esbuild@0.17.18\node_modules\esbuild\lib\main.js:1636
  let error = new Error(`${text}${summary}`);
              ^

Error: Build failed with 2 errors:
vite-plugin-wasm-namespace:F:\repos\runts\node_modules\.pnpm\vscode-oniguruma@1.7.0\node_modules\vscode-oniguruma\release\onig.wasm:43:160: ERROR: Could not resolve "env"      
vite-plugin-wasm-namespace:F:\repos\runts\node_modules\.pnpm\vscode-oniguruma@1.7.0\node_modules\vscode-oniguruma\release\onig.wasm:44:51: ERROR: Could not resolve "wasi_snapshot_preview1"
    at failureErrorWithLog (F:\repos\runts\node_modules\.pnpm\esbuild@0.17.18\node_modules\esbuild\lib\main.js:1636:15)
    at F:\repos\runts\node_modules\.pnpm\esbuild@0.17.18\node_modules\esbuild\lib\main.js:1048:25
    at F:\repos\runts\node_modules\.pnpm\esbuild@0.17.18\node_modules\esbuild\lib\main.js:1512:9
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5) {
  errors: [
    {
      detail: undefined,
      id: '',
      location: {
        column: 160,
        file: 'vite-plugin-wasm-namespace:F:\\repos\\runts\\node_modules\\.pnpm\\vscode-oniguruma@1.7.0\\node_modules\\vscode-oniguruma\\release\\onig.wasm',
        length: 5,
        line: 43,
        lineText: 'import { emscripten_memcpy_big as __vite__wasmImport_0_0, emscripten_get_now as __vite__wasmImport_0_1, emscripten_resize_heap as __vite__wasmImport_0_2 } from "env";',
        namespace: '',
        suggestion: ''
      },
      notes: [
        {
          location: null,
          text: 'You can mark the path "env" as external to exclude it from the bundle, which will remove this error.'
        }
      ],
      pluginName: '',
      text: 'Could not resolve "env"'
    },
    {
      detail: undefined,
      id: '',
      location: {
        column: 51,
        file: 'vite-plugin-wasm-namespace:F:\\repos\\runts\\node_modules\\.pnpm\\vscode-oniguruma@1.7.0\\node_modules\\vscode-oniguruma\\release\\onig.wasm',
        length: 24,
        line: 44,
        lineText: 'import { fd_write as __vite__wasmImport_1_0 } from "wasi_snapshot_preview1";',
        namespace: '',
        suggestion: ''
      },
      notes: [
        {
          location: null,
          text: 'You can mark the path "wasi_snapshot_preview1" as external to exclude it from the bundle, which will remove this error.'
        }
      ],
      pluginName: '',
      text: 'Could not resolve "wasi_snapshot_preview1"'
    }
  ],
  warnings: []
}

Node.js v19.2.0
 ELIFECYCLE  Command failed with exit code 1.

here's the code:

// eslint-disable-next-line simple-import-sort/imports
import { initialize } from "vscode/services";
// eslint-disable-next-line simple-import-sort/imports
import * as monaco from "monaco-editor";
import getTextMateServiceOverride from "vscode/service-override/textmate";
import MonacoEditor, {
  type EditorProps as MonacoEditorProps,
  loader,
  type Monaco,
} from "@monaco-editor/react";
import { type editor } from "monaco-editor";
import { forwardRef } from "react";

await initialize({
  ...getTextMateServiceOverride(),
});

loader.config({ monaco });
kaisalmen commented 1 year ago

@CGNonofr even with the latest release 1.78.7 this does not run with pnpm 8. yarn and npm works event with multiple install calls and manual treemending script invocation!

{
  "name": "monaco-languagclient-pnpm",
  "dependencies": {
    "monaco-languageclient": "6.0.3-next.1",
    "vscode": "npm:@codingame/monaco-vscode-api@~1.78.7"
  },
  "scripts": {
    "tester": "monaco-treemending"
  }
}

I have released a next version of monaco-languageclient that depends on the latest version and includes the "postinstall": "monaco-treemending" . pnpm failes to install. If you use pnpm install --ignore-scripts it will succeed with the installation, but if you trigger the script manually with with pnpm run tester or even with regular npm npm run tester it will fail. This must be due to directory structure and all the (hard) cross links. It screws up the patching process. 🤷‍♂️

CGNonofr commented 1 year ago

Do we have a solution for it? I don't know pnpm at all

kaisalmen commented 1 year ago

I have installed pnpm for the first time ever today, so I am as close to be an expert on it as you are. 🙂 Only idea I have is to look if monacoEsmDirectory resolution in the treemending can be improved somehow (due to links).

kaisalmen commented 1 year ago

@MrMarble pnpm related tree-mending issues should be resolved with monaco-vscode-api@1.78.8 and with monaco-languageclient@6.0.3. It depends on the latest version.

kaisalmen commented 1 year ago

@MrMarble FYI, I added yarn and pnpm verification packages into the monaco-languageclient repo, see https://github.com/TypeFox/monaco-languageclient#pure-bundler-verification

kaisalmen commented 1 year ago

@CGNonofr do you see a reason to keep this open?

CGNonofr commented 1 year ago

Not really, it was still looking like an open discussion. If you don't agree, feel free to close it :)

Sruthisreem commented 1 year ago

Hi i'm installing a dependency, which has a "postinstall": "monaco-treemending" . when I do Yarn install, build is failed with an error (I did removed node_modules folder before install) Mac OS: ventura 13.4 node : v16.18.0 yarn: 3.2.1

Screenshot 2023-07-03 at 13 50 46 Screenshot 2023-07-03 at 13 59 19
CGNonofr commented 1 year ago

The issue is probably a monaco-editor version mismatch

char0n commented 1 year ago

SwaggerEditor@5.0.0-alpha.67 comes with following combo:

    "monaco-editor": "^0.39.0",
    "vscode": "npm:@codingame/monaco-vscode-api@~1.79.6"

But yarn lock might be overriding it. @Sruthisreem make sure your yarn lock contains versions above.

Sruthisreem commented 1 year ago

@char0n The yarn lock contains "monaco-editor" : 0.39.0

Screenshot 2023-07-03 at 15 17 44
char0n commented 1 year ago

Regarding the yarn issue; here are my latest investigation notes: https://github.com/swagger-api/swagger-editor/issues/4264#issuecomment-1628414615

char0n commented 1 year ago

Regarding the yarn issue:

yarn@3.2.1 had an issue with postinstall lifecycle hook. It was called when monaco-editor now still not present in node_modules. Latest yarn@3.6.1 works correctly.

CGNonofr commented 1 year ago

@kaisalmen random idea:

Since we now publish tons of package, we can publish an additional one with the patched version of monaco-editor, as an alternative to the treemending script (it would require a npm alias)

What do you think?

kaisalmen commented 1 year ago

@CGNonofr Yes, good idea. That can be the easier alternative for some people.

CGNonofr commented 1 year ago

The alternative way was released: instead of the treemending script, you can now put "monaco-editor": "npm:@codingame/monaco-editor-treemended@<version>" in your dependencies

It may require to force override the version in the overrides section as well to prevent conflict with other libraries having monaco-editor as dependency