jupyterlab / extension-cookiecutter-ts

A cookiecutter recipe for JupyterLab extensions in Typescript
BSD 3-Clause "New" or "Revised" License
179 stars 88 forks source link

Yarn errors during "Preparing editable metadata" #306

Closed jwindgassen closed 1 year ago

jwindgassen commented 1 year ago

Description

I am trying to create a JupyterLab extension generated with this template, but as soon as I try to execute pip install -e . (as described in the README), I get yarn errors during the Preparing editable metadata step.

As far as I can tell, yarn has trouble installing downloading/installing packages on our system, but I was unable to find any reason why: All related directories and files are read- and writable and there should be enough diskspace. It might also be an error inside hatch-jupyter-builder or inside the Node.js installation on our system, but I am not able to pinpoint the main culprit for the error.

Options used during cookiecutter:

Context

System: JUWELS Booster (Rocky Linux) JupyterLab Version: 3.6.3 Python Version: 3.10.4 Node.js Version: 16.15.1 Yarn Version: 1.22.19

Log

Command Line Output
Processing /p/home/jusers/windgassen1/juwels/trame/jupyter_viz_extension
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Installing backend dependencies ... done
  Preparing metadata (pyproject.toml) ... error
  error: subprocess-exited-with-error

  × Preparing metadata (pyproject.toml) did not run successfully.
  │ exit code: 1
  ╰─> [38 lines of output]
      INFO:hatch_jupyter_builder.utils:Running jupyter-builder
      INFO:hatch_jupyter_builder.utils:Building with hatch_jupyter_builder.npm_builder
      INFO:hatch_jupyter_builder.utils:With kwargs: {'build_cmd': 'build:prod', 'npm': ['jlpm']}
      INFO:hatch_jupyter_builder.utils:Installing build dependencies with npm.  This may take a while...
      INFO:hatch_jupyter_builder.utils:> /tmp/pip-build-env-yqmk_0cc/overlay/bin/jlpm install
      yarn install v1.21.1
      info No lockfile found.
      [1/4] Resolving packages...
      warning @jupyterlab/application > @jupyterlab/apputils > url > querystring@0.2.0: The querystring API is considered Legacy. new code should use the URLSearchParams API instead.
      warning @jupyterlab/application > @jupyterlab/ui-components > @blueprintjs/core > popper.js@1.16.1: You can find the new Popper v2 at @popperjs/core, this package is dedicated to the legacy v1
      warning @jupyterlab/application > @jupyterlab/ui-components > @blueprintjs/core > react-popper > popper.js@1.16.1: You can find the new Popper v2 at @popperjs/core, this package is dedicated to the legacy v1
      warning @jupyterlab/builder > terser-webpack-plugin > cacache > @npmcli/move-file@1.1.2: This functionality has been moved to @npmcli/fs
      [2/4] Fetching packages...
      error An unexpected error occurred: "https://registry.yarnpkg.com/@typescript-eslint/experimental-utils/-/experimental-utils-4.33.0.tgz: Unknown system error -122: Unknown system error -122, open '/p/home/jusers/windgassen1/juwels/.cache/yarn/v6/npm-@typescript-eslint-experimental-utils-4.33.0-6f2a786a4209fa2222989e9380b5331b2810f7fd-integrity/node_modules/@typescript-eslint/experimental-utils/.yarn-tarball.tgz'".
      info If you think this is a bug, please open a bug report with the information provided in "/p/home/jusers/windgassen1/juwels/trame/jupyter_viz_extension/yarn-error.log".
      info Visit https://yarnpkg.com/en/docs/cli/install for documentation about this command.
      Traceback (most recent call last):
        File "/p/project/dems/windgassen1/jupyterlab/lib/python3.10/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 353, in 
          main()
        File "/p/project/dems/windgassen1/jupyterlab/lib/python3.10/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 335, in main
          json_out['return_val'] = hook(**hook_input['kwargs'])
        File "/p/project/dems/windgassen1/jupyterlab/lib/python3.10/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 152, in prepare_metadata_for_build_wheel
          whl_basename = backend.build_wheel(metadata_directory, config_settings)
        File "/tmp/pip-build-env-yqmk_0cc/overlay/lib/python3.10/site-packages/hatchling/build.py", line 56, in build_wheel
          return os.path.basename(next(builder.build(wheel_directory, ['standard'])))
        File "/tmp/pip-build-env-yqmk_0cc/overlay/lib/python3.10/site-packages/hatchling/builders/plugin/interface.py", line 150, in build
          build_hook.initialize(version, build_data)
        File "/tmp/pip-build-env-yqmk_0cc/normal/lib/python3.10/site-packages/hatch_jupyter_builder/plugin.py", line 87, in initialize
          raise e
        File "/tmp/pip-build-env-yqmk_0cc/normal/lib/python3.10/site-packages/hatch_jupyter_builder/plugin.py", line 82, in initialize
          build_func(self.target_name, version, **build_kwargs)
        File "/tmp/pip-build-env-yqmk_0cc/normal/lib/python3.10/site-packages/hatch_jupyter_builder/utils.py", line 113, in npm_builder
          run([*npm_cmd, "install"], cwd=str(abs_path))
        File "/tmp/pip-build-env-yqmk_0cc/normal/lib/python3.10/site-packages/hatch_jupyter_builder/utils.py", line 229, in run
          return subprocess.check_call(cmd, **kwargs)
        File "/p/software/juwelsbooster/stages/2023/software/Python/3.10.4-GCCcore-11.3.0/lib/python3.10/subprocess.py", line 369, in check_call
          raise CalledProcessError(retcode, cmd)
      subprocess.CalledProcessError: Command '['/tmp/pip-build-env-yqmk_0cc/overlay/bin/jlpm', 'install']' returned non-zero exit status 1.
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
error: metadata-generation-failed

× Encountered error while generating package metadata.
╰─> See above for output.

note: This is an issue with the package mentioned above, not pip.
hint: See above for details.
yarn-error.log
Arguments: 
  /p/software/juwelsbooster/stages/2023/software/nodejs/16.15.1-GCCcore-11.3.0/bin/node /tmp/pip-build-env-yqmk_0cc/overlay/lib/python3.10/site-packages/jupyterlab/staging/yarn.js install

PATH: 
  /tmp/pip-build-env-yqmk_0cc/overlay/bin:/tmp/pip-build-env-yqmk_0cc/normal/bin:/p/project/dems/windgassen1/jupyterlab/bin:/p/software/juwelsbooster/stages/2023/software/yarn/1.22.19-GCCcore-11.3.0:/p/software/juwelsbooster/stages/2023/software/yarn/1.22.19-GCCcore-11.3.0/bin:/p/software/juwelsbooster/stages/2023/software/nodejs/16.15.1-GCCcore-11.3.0/bin:/p/software/juwelsbooster/stages/2023/software/Python/3.10.4-GCCcore-11.3.0/bin:/p/software/juwelsbooster/stages/2023/software/OpenSSL/1.1/bin:/p/software/juwelsbooster/stages/2023/software/XZ/5.2.5-GCCcore-11.3.0/bin:/p/software/juwelsbooster/stages/2023/software/SQLite/3.38.3-GCCcore-11.3.0/bin:/p/software/juwelsbooster/stages/2023/software/Tcl/8.6.12-GCCcore-11.3.0/bin:/p/software/juwelsbooster/stages/2023/software/ncurses/6.3-GCCcore-11.3.0/bin:/p/software/juwelsbooster/stages/2023/software/bzip2/1.0.8-GCCcore-11.3.0/bin:/p/software/juwelsbooster/stages/2023/software/binutils/2.38-GCCcore-11.3.0/bin:/p/software/juwelsbooster/stages/2023/software/GCCcore/11.3.0/bin:/p/home/jusers/windgassen1/juwels/.local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/opt/ddn/ime/bin:/opt/jsc/bin:/usr/local/jsc/bin:/opt/parastation/bin:/p/software/juwelsbooster/bin

Yarn version: 
  1.21.1

Node version: 
  16.15.1

Platform: 
  linux x64

Trace: 
  Error: Unknown system error -122: Unknown system error -122, open '/p/home/jusers/windgassen1/juwels/.cache/yarn/v6/npm-@typescript-eslint-experimental-utils-4.33.0-6f2a786a4209fa2222989e9380b5331b2810f7fd-integrity/node_modules/@typescript-eslint/experimental-utils/.yarn-tarball.tgz'

npm manifest: 
  {
    "name": "jupyter-viz-extension",
    "version": "0.1.0",
    "description": "A JupyterLab extension for easy-to-use browser-based visualization with ParaView trame",
    "keywords": [
      "jupyter",
      "jupyterlab",
      "jupyterlab-extension"
    ],
    "homepage": "https://github.com/jwindgassen/jupyter-viz-extension",
    "bugs": {
      "url": "https://github.com/jwindgassen/jupyter-viz-extension/issues"
    },
    "license": "BSD-3-Clause",
    "author": {
      "name": "Jonathan Windgassen",
      "email": "j.windgassen@fz-juelich.de"
    },
    "files": [
      "lib/**/*.{d.ts,eot,gif,html,jpg,js,js.map,json,png,svg,woff2,ttf}",
      "style/**/*.{css,js,eot,gif,html,jpg,json,png,svg,woff2,ttf}"
    ],
    "main": "lib/index.js",
    "types": "lib/index.d.ts",
    "style": "style/index.css",
    "repository": {
      "type": "git",
      "url": "https://github.com/jwindgassen/jupyter-viz-extension.git"
    },
    "scripts": {
      "build": "jlpm build:lib && jlpm build:labextension:dev",
      "build:prod": "jlpm clean && jlpm build:lib:prod && jlpm build:labextension",
      "build:labextension": "jupyter labextension build .",
      "build:labextension:dev": "jupyter labextension build --development True .",
      "build:lib": "tsc --sourceMap",
      "build:lib:prod": "tsc",
      "clean": "jlpm clean:lib",
      "clean:lib": "rimraf lib tsconfig.tsbuildinfo",
      "clean:lintcache": "rimraf .eslintcache .stylelintcache",
      "clean:labextension": "rimraf jupyter_viz_extension/labextension jupyter_viz_extension/_version.py",
      "clean:all": "jlpm clean:lib && jlpm clean:labextension && jlpm clean:lintcache",
      "eslint": "jlpm eslint:check --fix",
      "eslint:check": "eslint . --cache --ext .ts,.tsx",
      "install:extension": "jlpm build",
      "lint": "jlpm stylelint && jlpm prettier && jlpm eslint",
      "lint:check": "jlpm stylelint:check && jlpm prettier:check && jlpm eslint:check",
      "prettier": "jlpm prettier:base --write --list-different",
      "prettier:base": "prettier \"**/*{.ts,.tsx,.js,.jsx,.css,.json,.md}\"",
      "prettier:check": "jlpm prettier:base --check",
      "stylelint": "jlpm stylelint:check --fix",
      "stylelint:check": "stylelint --cache \"style/**/*.css\"",
      "watch": "run-p watch:src watch:labextension",
      "watch:src": "tsc -w",
      "watch:labextension": "jupyter labextension watch ."
    },
    "dependencies": {
      "@jupyterlab/application": "^3.1.0",
      "@jupyterlab/coreutils": "^5.1.0",
      "@jupyterlab/services": "^6.1.0"
    },
    "devDependencies": {
      "@jupyterlab/builder": "^3.1.0",
      "@typescript-eslint/eslint-plugin": "^4.8.1",
      "@typescript-eslint/parser": "^4.8.1",
      "eslint": "^7.14.0",
      "eslint-config-prettier": "^6.15.0",
      "eslint-plugin-prettier": "^3.1.4",
      "mkdirp": "^1.0.3",
      "npm-run-all": "^4.1.5",
      "prettier": "^2.1.1",
      "rimraf": "^3.0.2",
      "stylelint": "^14.3.0",
      "stylelint-config-prettier": "^9.0.4",
      "stylelint-config-recommended": "^6.0.0",
      "stylelint-config-standard": "~24.0.0",
      "stylelint-prettier": "^2.0.0",
      "typescript": "~4.1.3"
    },
    "sideEffects": [
      "style/*.css",
      "style/index.js"
    ],
    "styleModule": "style/index.js",
    "publishConfig": {
      "access": "public"
    },
    "jupyterlab": {
      "discovery": {
        "server": {
          "managers": [
            "pip"
          ],
          "base": {
            "name": "jupyter_viz_extension"
          }
        }
      },
      "extension": true,
      "outputDir": "jupyter_viz_extension/labextension"
    },
    "eslintIgnore": [
      "node_modules",
      "dist",
      "coverage",
      "**/*.d.ts"
    ],
    "eslintConfig": {
      "extends": [
        "eslint:recommended",
        "plugin:@typescript-eslint/eslint-recommended",
        "plugin:@typescript-eslint/recommended",
        "plugin:prettier/recommended"
      ],
      "parser": "@typescript-eslint/parser",
      "parserOptions": {
        "project": "tsconfig.json",
        "sourceType": "module"
      },
      "plugins": [
        "@typescript-eslint"
      ],
      "rules": {
        "@typescript-eslint/naming-convention": [
          "error",
          {
            "selector": "interface",
            "format": [
              "PascalCase"
            ],
            "custom": {
              "regex": "^I[A-Z]",
              "match": true
            }
          }
        ],
        "@typescript-eslint/no-unused-vars": [
          "warn",
          {
            "args": "none"
          }
        ],
        "@typescript-eslint/no-explicit-any": "off",
        "@typescript-eslint/no-namespace": "off",
        "@typescript-eslint/no-use-before-define": "off",
        "@typescript-eslint/quotes": [
          "error",
          "single",
          {
            "avoidEscape": true,
            "allowTemplateLiterals": false
          }
        ],
        "curly": [
          "error",
          "all"
        ],
        "eqeqeq": "error",
        "prefer-arrow-callback": "error"
      }
    },
    "prettier": {
      "singleQuote": true,
      "trailingComma": "none",
      "arrowParens": "avoid",
      "endOfLine": "auto"
    },
    "stylelint": {
      "extends": [
        "stylelint-config-recommended",
        "stylelint-config-standard",
        "stylelint-prettier/recommended"
      ],
      "rules": {
        "property-no-vendor-prefix": null,
        "selector-no-vendor-prefix": null,
        "value-no-vendor-prefix": null
      }
    }
  }

yarn manifest: 
  No manifest

Lockfile: 
  No lockfile
welcome[bot] commented 1 year ago

Thank you for opening your first issue in this project! Engagement like this is essential for open source projects! :hugs:
If you haven't done so already, check out Jupyter's Code of Conduct. Also, please try to follow the issue template as it helps other other community members to contribute more effectively. welcome You can meet the other Jovyans by joining our Discourse forum. There is also an intro thread there where you can stop by and say Hi! :wave:
Welcome to the Jupyter community! :tada:

fcollonval commented 1 year ago

@jwindgassen the error trace is quite verbose. But the root cause of your error is:

error https://registry.yarnpkg.com/@typescript-eslint/typescript-estree/-/typescript-estree-4.33.0.tgz: Extracting tar content of undefined failed, the file appears to be corrupt: "Unknown system error -122: Unknown system error -122, open '/p/home/jusers/windgassen1/juwels/.cache/yarn/v6/[npm-@typescript-eslint-typescript-estree-4.33.0-0dfb51c2908f68c5c08d82aefeaf166a17c24609-integrity](mailto:npm-@typescript-eslint-typescript-estree-4.33.0-0dfb51c2908f68c5c08d82aefeaf166a17c24609-integrity)/node_modules/@typescript-eslint/typescript-estree/dist/ts-estree/ts-nodes.d.ts.map'"

It seems like a transient error. Did you hit the error by retrying?

jwindgassen commented 1 year ago

All tries I did ended in some error like that, but the exact package might differ. I took a look at the .cache folder, but yarn/v6/<package-name>/node_modules is empty. So that is probably the reason the opening fails. But I can't see any immediate reason for the directory being empty.

Note: I changed the Versions I am using and adjusted the log accordingly (But the error is still the same). I added the yarn-error.log above, as it is mentioned now.

fcollonval commented 1 year ago

Have you tried cleaning your cache: https://classic.yarnpkg.com/en/docs/cli/cache#toc-yarn-cache-clean

jwindgassen commented 1 year ago

Already did, yes. I also made sure the directories are writable and that I have enough space on disk.

fcollonval commented 1 year ago

Looking again at your error trace, you have an OS error: Unknown system error -122

Googling it: https://www.google.com/search?q=Linux+%22Unknown+system+error+-122%22 points most of the time toward having too little disk space.

jwindgassen commented 1 year ago

I found the problem: It was not really a disk-space issue, but the number of inodes that were problematic. I think I had ~30k inodes left, which I thought was plenty, but I just vastly underestimated the number of inodes required. When I redirected the yarn cache directory to another location with enough inodes free, the error disappeared!

fcollonval commented 1 year ago

Thanks @jwindgassen for coming back sharing the final root cause (and fix).