Sage-Bionetworks / sage-monorepo

Where OpenChallenges, Schematic, and other Sage open source apps are built
https://sage-bionetworks.github.io/sage-monorepo/
Apache License 2.0
23 stars 12 forks source link

[Bug] Fix `nx run schematic-api:prepare-python` #1784

Closed tschaffter closed 1 year ago

tschaffter commented 1 year ago

Is there an existing issue for this?

What projects are you seeing the problem on?

Schematic

Current behavior

Preparing the Python environment for the project schematic-api is now failing:

$ nx run schematic-api:prepare-python
...
• Installing uritemplate (3.0.1)
  • Installing werkzeug (1.0.1)
  • Installing widgetsnbextension (4.0.7)
  • Installing wrapt (1.15.0)
  CalledProcessError
  Command '['/workspaces/sage-monorepo/apps/schematic/api/.venv/bin/python', '-m', 'pip', 'install', '--use-pep517', '--disable-pip-version-check', '--isolated', '--no-input', '--prefix', '/workspaces/sage-monorepo/apps/schematic/api/.venv', '--no-deps', '/home/vscode/.cache/pypoetry/artifacts/b6/23/45/f5dfdd6e8ba0f620504858ddeb20b47f50b03d0c4b18f873f6575d2e78/PyYAML-5.4.1.tar.gz']' returned non-zero exit status 1.
  at /usr/lib/python3.9/subprocess.py:528 in run
       524│             # We don't call process.wait() as .__exit__ does that for us.
       525│             raise
       526│         retcode = process.poll()
       527│         if check and retcode:
    →  528│             raise CalledProcessError(retcode, process.args,
       529│                                      output=stdout, stderr=stderr)
       530│     return CompletedProcess(process.args, retcode, stdout, stderr)
       531│ 
       532│ 
The following error occurred when trying to handle this error:
  EnvCommandError
  Command ['/workspaces/sage-monorepo/apps/schematic/api/.venv/bin/python', '-m', 'pip', 'install', '--use-pep517', '--disable-pip-version-check', '--isolated', '--no-input', '--prefix', '/workspaces/sage-monorepo/apps/schematic/api/.venv', '--no-deps', '/home/vscode/.cache/pypoetry/artifacts/b6/23/45/f5dfdd6e8ba0f620504858ddeb20b47f50b03d0c4b18f873f6575d2e78/PyYAML-5.4.1.tar.gz'] errored with the following return code 1, and output: 
  Processing /home/vscode/.cache/pypoetry/artifacts/b6/23/45/f5dfdd6e8ba0f620504858ddeb20b47f50b03d0c4b18f873f6575d2e78/PyYAML-5.4.1.tar.gz
    Installing build dependencies: started
    Installing build dependencies: finished with status 'done'
    Getting requirements to build wheel: started
    Getting requirements to build wheel: finished with status 'error'
    error: subprocess-exited-with-error

    × Getting requirements to build wheel did not run successfully.
    │ exit code: 1
    ╰─> [62 lines of output]
        /tmp/pip-build-env-2gtgyxup/overlay/lib/python3.10/site-packages/setuptools/config/setupcfg.py:293: _DeprecatedConfig: Deprecated config in `setup.cfg`
        !!

                ********************************************************************************
                The license_file parameter is deprecated, use license_files instead.

                By 2023-Oct-30, you need to update your project and remove deprecated calls
                or your builds will no longer be supported.

                See https://setuptools.pypa.io/en/latest/userguide/declarative_config.html for details.
                ********************************************************************************

        !!
          parsed = self.parsers.get(option_name, lambda x: x)(value)
        running egg_info
        writing lib3/PyYAML.egg-info/PKG-INFO
        writing dependency_links to lib3/PyYAML.egg-info/dependency_links.txt
        writing top-level names to lib3/PyYAML.egg-info/top_level.txt
        Traceback (most recent call last):
          File "/workspaces/sage-monorepo/apps/schematic/api/.venv/lib/python3.10/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 353, in <module>
            main()
          File "/workspaces/sage-monorepo/apps/schematic/api/.venv/lib/python3.10/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 335, in main
            json_out['return_val'] = hook(**hook_input['kwargs'])
          File "/workspaces/sage-monorepo/apps/schematic/api/.venv/lib/python3.10/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 118, in get_requires_for_build_wheel
            return hook(config_settings)
          File "/tmp/pip-build-env-2gtgyxup/overlay/lib/python3.10/site-packages/setuptools/build_meta.py", line 341, in get_requires_for_build_wheel
            return self._get_build_requires(config_settings, requirements=['wheel'])
          File "/tmp/pip-build-env-2gtgyxup/overlay/lib/python3.10/site-packages/setuptools/build_meta.py", line 323, in _get_build_requires
            self.run_setup()
          File "/tmp/pip-build-env-2gtgyxup/overlay/lib/python3.10/site-packages/setuptools/build_meta.py", line 338, in run_setup
            exec(code, locals())
          File "<string>", line 271, in <module>
          File "/tmp/pip-build-env-2gtgyxup/overlay/lib/python3.10/site-packages/setuptools/__init__.py", line 107, in setup
            return distutils.core.setup(**attrs)
          File "/tmp/pip-build-env-2gtgyxup/overlay/lib/python3.10/site-packages/setuptools/_distutils/core.py", line 185, in setup
            return run_commands(dist)
          File "/tmp/pip-build-env-2gtgyxup/overlay/lib/python3.10/site-packages/setuptools/_distutils/core.py", line 201, in run_commands
            dist.run_commands()
          File "/tmp/pip-build-env-2gtgyxup/overlay/lib/python3.10/site-packages/setuptools/_distutils/dist.py", line 969, in run_commands
            self.run_command(cmd)
          File "/tmp/pip-build-env-2gtgyxup/overlay/lib/python3.10/site-packages/setuptools/dist.py", line 1234, in run_command
            super().run_command(command)
          File "/tmp/pip-build-env-2gtgyxup/overlay/lib/python3.10/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
            cmd_obj.run()
          File "/tmp/pip-build-env-2gtgyxup/overlay/lib/python3.10/site-packages/setuptools/command/egg_info.py", line 314, in run
            self.find_sources()
          File "/tmp/pip-build-env-2gtgyxup/overlay/lib/python3.10/site-packages/setuptools/command/egg_info.py", line 322, in find_sources
            mm.run()
          File "/tmp/pip-build-env-2gtgyxup/overlay/lib/python3.10/site-packages/setuptools/command/egg_info.py", line 551, in run
            self.add_defaults()
          File "/tmp/pip-build-env-2gtgyxup/overlay/lib/python3.10/site-packages/setuptools/command/egg_info.py", line 589, in add_defaults
            sdist.add_defaults(self)
          File "/tmp/pip-build-env-2gtgyxup/overlay/lib/python3.10/site-packages/setuptools/command/sdist.py", line 104, in add_defaults
            super().add_defaults()
          File "/tmp/pip-build-env-2gtgyxup/overlay/lib/python3.10/site-packages/setuptools/_distutils/command/sdist.py", line 251, in add_defaults
            self._add_defaults_ext()
          File "/tmp/pip-build-env-2gtgyxup/overlay/lib/python3.10/site-packages/setuptools/_distutils/command/sdist.py", line 336, in _add_defaults_ext
            self.filelist.extend(build_ext.get_source_files())
          File "<string>", line 201, in get_source_files
          File "/tmp/pip-build-env-2gtgyxup/overlay/lib/python3.10/site-packages/setuptools/_distutils/cmd.py", line 107, in __getattr__
            raise AttributeError(attr)
        AttributeError: cython_sources
        [end of output]

    note: This error originates from a subprocess, and is likely not a problem with pip.
  error: subprocess-exited-with-error

  × Getting requirements to build wheel did not run successfully.
  │ exit code: 1
  ╰─> See above for output.

  note: This error originates from a subprocess, and is likely not a problem with pip.

  at /etc/poetry/venv/lib/python3.9/site-packages/poetry/utils/env.py:1540 in _run
      1536│                 output = subprocess.check_output(
      1537│                     command, stderr=subprocess.STDOUT, env=env, **kwargs
      1538│                 )
      1539│         except CalledProcessError as e:
    → 1540│             raise EnvCommandError(e, input=input_)
      1541│ 
      1542│         return decode(output)
      1543│ 
      1544│     def execute(self, bin: str, *args: str, **kwargs: Any) -> int:
The following error occurred when trying to handle this error:
  PoetryException
  Failed to install /home/vscode/.cache/pypoetry/artifacts/b6/23/45/f5dfdd6e8ba0f620504858ddeb20b47f50b03d0c4b18f873f6575d2e78/PyYAML-5.4.1.tar.gz
  at /etc/poetry/venv/lib/python3.9/site-packages/poetry/utils/pip.py:58 in pip_install
       54│ 
       55│     try:
       56│         return environment.run_pip(*args)
       57│     except EnvCommandError as e:
    →  58│         raise PoetryException(f"Failed to install {path.as_posix()}") from e
       59│ 
 >  NX   ERROR: Something went wrong in run-commands - Command failed: poetry install --with prod,dev
   Pass --verbose to see the stacktrace.

cc @andrewelamb @linglp

Expected behavior

The command should successfully create the Python venv for this project.

Anything else?

Remove the folder apps/schematic/api/.venv before running nx prepare-python schematic-api otherwise the existing .venv won't be created and the error will not be thrown.

Commit ID

923a9a3e74e6f106997ff58f107d14e89a4b1214

Are you developing inside the dev container?

Code of Conduct

tschaffter commented 1 year ago

Installing pyyaml seems to trigger the issue.

• Installing pyyaml (5.4.1): Failed

  CalledProcessError

  Command '['/workspaces/sage-monorepo/apps/schematic/api/.venv/bin/python', '-m', 'pip', 'install', '--use-pep517', '--disable-pip-version-check', '--isolated', '--no-input', '--prefix', '/workspaces/sage-monorepo/apps/schematic/api/.venv', '--no-deps', '/home/vscode/.cache/pypoetry/artifacts/b6/23/45/f5dfdd6e8ba0f620504858ddeb20b47f50b03d0c4b18f873f6575d2e78/PyYAML-5.4.1.tar.gz']' returned non-zero exit status 1.

  at /usr/lib/python3.9/subprocess.py:528 in run
       524│             # We don't call process.wait() as .__exit__ does that for us.
       525│             raise
       526│         retcode = process.poll()
       527│         if check and retcode:
    →  528│             raise CalledProcessError(retcode, process.args,
       529│                                      output=stdout, stderr=stderr)
       530│     return CompletedProcess(process.args, retcode, stdout, stderr)
       531│ 
       532│ 

The following error occurred when trying to handle this error:

  EnvCommandError

  Command ['/workspaces/sage-monorepo/apps/schematic/api/.venv/bin/python', '-m', 'pip', 'install', '--use-pep517', '--disable-pip-version-check', '--isolated', '--no-input', '--prefix', '/workspaces/sage-monorepo/apps/schematic/api/.venv', '--no-deps', '/home/vscode/.cache/pypoetry/artifacts/b6/23/45/f5dfdd6e8ba0f620504858ddeb20b47f50b03d0c4b18f873f6575d2e78/PyYAML-5.4.1.tar.gz'] errored with the following return code 1, and output: 
  Processing /home/vscode/.cache/pypoetry/artifacts/b6/23/45/f5dfdd6e8ba0f620504858ddeb20b47f50b03d0c4b18f873f6575d2e78/PyYAML-5.4.1.tar.gz
    Installing build dependencies: started
    Installing build dependencies: finished with status 'done'
    Getting requirements to build wheel: started
    Getting requirements to build wheel: finished with status 'error'
    error: subprocess-exited-with-error

    × Getting requirements to build wheel did not run successfully.
    │ exit code: 1
    ╰─> [62 lines of output]
        /tmp/pip-build-env-e7lkk8yz/overlay/lib/python3.10/site-packages/setuptools/config/setupcfg.py:293: _DeprecatedConfig: Deprecated config in `setup.cfg`
        !!

                ********************************************************************************
                The license_file parameter is deprecated, use license_files instead.

                By 2023-Oct-30, you need to update your project and remove deprecated calls
                or your builds will no longer be supported.

                See https://setuptools.pypa.io/en/latest/userguide/declarative_config.html for details.
                ********************************************************************************

        !!
          parsed = self.parsers.get(option_name, lambda x: x)(value)
        running egg_info
        writing lib3/PyYAML.egg-info/PKG-INFO
        writing dependency_links to lib3/PyYAML.egg-info/dependency_links.txt
        writing top-level names to lib3/PyYAML.egg-info/top_level.txt
        Traceback (most recent call last):
          File "/workspaces/sage-monorepo/apps/schematic/api/.venv/lib/python3.10/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 353, in <module>
            main()
          File "/workspaces/sage-monorepo/apps/schematic/api/.venv/lib/python3.10/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 335, in main
            json_out['return_val'] = hook(**hook_input['kwargs'])
          File "/workspaces/sage-monorepo/apps/schematic/api/.venv/lib/python3.10/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 118, in get_requires_for_build_wheel
            return hook(config_settings)
          File "/tmp/pip-build-env-e7lkk8yz/overlay/lib/python3.10/site-packages/setuptools/build_meta.py", line 341, in get_requires_for_build_wheel
            return self._get_build_requires(config_settings, requirements=['wheel'])
          File "/tmp/pip-build-env-e7lkk8yz/overlay/lib/python3.10/site-packages/setuptools/build_meta.py", line 323, in _get_build_requires
            self.run_setup()
          File "/tmp/pip-build-env-e7lkk8yz/overlay/lib/python3.10/site-packages/setuptools/build_meta.py", line 338, in run_setup
            exec(code, locals())
          File "<string>", line 271, in <module>
          File "/tmp/pip-build-env-e7lkk8yz/overlay/lib/python3.10/site-packages/setuptools/__init__.py", line 107, in setup
            return distutils.core.setup(**attrs)
          File "/tmp/pip-build-env-e7lkk8yz/overlay/lib/python3.10/site-packages/setuptools/_distutils/core.py", line 185, in setup
            return run_commands(dist)
          File "/tmp/pip-build-env-e7lkk8yz/overlay/lib/python3.10/site-packages/setuptools/_distutils/core.py", line 201, in run_commands
            dist.run_commands()
          File "/tmp/pip-build-env-e7lkk8yz/overlay/lib/python3.10/site-packages/setuptools/_distutils/dist.py", line 969, in run_commands
            self.run_command(cmd)
          File "/tmp/pip-build-env-e7lkk8yz/overlay/lib/python3.10/site-packages/setuptools/dist.py", line 1234, in run_command
            super().run_command(command)
          File "/tmp/pip-build-env-e7lkk8yz/overlay/lib/python3.10/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
            cmd_obj.run()
          File "/tmp/pip-build-env-e7lkk8yz/overlay/lib/python3.10/site-packages/setuptools/command/egg_info.py", line 314, in run
            self.find_sources()
          File "/tmp/pip-build-env-e7lkk8yz/overlay/lib/python3.10/site-packages/setuptools/command/egg_info.py", line 322, in find_sources
            mm.run()
          File "/tmp/pip-build-env-e7lkk8yz/overlay/lib/python3.10/site-packages/setuptools/command/egg_info.py", line 551, in run
            self.add_defaults()
          File "/tmp/pip-build-env-e7lkk8yz/overlay/lib/python3.10/site-packages/setuptools/command/egg_info.py", line 589, in add_defaults
            sdist.add_defaults(self)
          File "/tmp/pip-build-env-e7lkk8yz/overlay/lib/python3.10/site-packages/setuptools/command/sdist.py", line 104, in add_defaults
            super().add_defaults()
          File "/tmp/pip-build-env-e7lkk8yz/overlay/lib/python3.10/site-packages/setuptools/_distutils/command/sdist.py", line 251, in add_defaults
            self._add_defaults_ext()
          File "/tmp/pip-build-env-e7lkk8yz/overlay/lib/python3.10/site-packages/setuptools/_distutils/command/sdist.py", line 336, in _add_defaults_ext
            self.filelist.extend(build_ext.get_source_files())
          File "<string>", line 201, in get_source_files
          File "/tmp/pip-build-env-e7lkk8yz/overlay/lib/python3.10/site-packages/setuptools/_distutils/cmd.py", line 107, in __getattr__
            raise AttributeError(attr)
        AttributeError: cython_sources
        [end of output]

    note: This error originates from a subprocess, and is likely not a problem with pip.
  error: subprocess-exited-with-error

  × Getting requirements to build wheel did not run successfully.
  │ exit code: 1
  ╰─> See above for output.

  note: This error originates from a subprocess, and is likely not a problem with pip.

  at /etc/poetry/venv/lib/python3.9/site-packages/poetry/utils/env.py:1540 in _run
      1536│                 output = subprocess.check_output(
      1537│                     command, stderr=subprocess.STDOUT, env=env, **kwargs
      1538│                 )
      1539│         except CalledProcessError as e:
    → 1540│             raise EnvCommandError(e, input=input_)
      1541│ 
      1542│         return decode(output)
      1543│ 
      1544│     def execute(self, bin: str, *args: str, **kwargs: Any) -> int:

The following error occurred when trying to handle this error:

  PoetryException

  Failed to install /home/vscode/.cache/pypoetry/artifacts/b6/23/45/f5dfdd6e8ba0f620504858ddeb20b47f50b03d0c4b18f873f6575d2e78/PyYAML-5.4.1.tar.gz

  at /etc/poetry/venv/lib/python3.9/site-packages/poetry/utils/pip.py:58 in pip_install
       54│ 
       55│     try:
       56│         return environment.run_pip(*args)
       57│     except EnvCommandError as e:
    →  58│         raise PoetryException(f"Failed to install {path.as_posix()}") from e
       59│ 

  • Installing tomli (2.0.1)
tschaffter commented 1 year ago

The command that fails is poetry install --with prod,dev.

The above command works fine when removing schematicpy = "23.1.1" from pyproject.toml and the lock file.

tschaffter commented 1 year ago

This issue was reported one hour ago to the GitHub repo of pyyaml.