search

facebookresearch / sapiens

High-resolution models for human tasks.
https://about.meta.com/realitylabs/codecavatars/sapiens/
Other
4.56k stars 261 forks source link

Develop Environment Cannot be setup #171

Closed Transfor-Cat closed 1 day ago

Transfor-Cat commented 1 week ago

I am trying to reproduce the reported result by using 'bash conda.sh'. However, conda finds lots of conflicts when installing packages. Can you provide a detailed environment setup file 'requiements.txt' which contains all package versions without conflicts?

rawalkhirodkar commented 1 week ago

@Transfor-Cat can you share the error log when using conda.sh?

Transfor-Cat commented 1 week ago

Thanks for your reply. The detailed logs are as follows: When installing libcom library, there is a compiling error

Emitting ninja build file /tmp/pip-install-g954630y/libcom_1675273f43764efcbff80639b110a4a3/build/temp.linux-x86_64-cpython-310/build.ninja...
      Compiling objects...
      Allowing ninja to set a default number of workers... (overridable by setting the environment variable MAX_JOBS=N)
      ninja: error: '/tmp/pip-install-g954630y/libcom_1675273f43764efcbff80639b110a4a3/libcom/image_harmonization/source/trilinear_cpp_torch1.11/src/trilinear_cuda.cpp', needed by '/tmp/pip-install-g954630y/libcom_1675273f43764efcbff80639b110a4a3/build/temp.linux-x86_64-cpython-310/tmp/pip-install-g954630y/libcom_1675273f43764efcbff80639b110a4a3/libcom/image_harmonization/source/trilinear_cpp_torch1.11/src/trilinear_cuda.o', missing and no known rule to make it
      Traceback (most recent call last):
        File "/root/miniconda3/envs/sapiens/lib/python3.10/site-packages/torch/utils/cpp_extension.py", line 2107, in _run_ninja_build
          subprocess.run(
        File "/root/miniconda3/envs/sapiens/lib/python3.10/subprocess.py", line 526, in run
          raise CalledProcessError(retcode, process.args,
      subprocess.CalledProcessError: Command '['ninja', '-v']' returned non-zero exit status 1.

      The above exception was the direct cause of the following exception:

      Traceback (most recent call last):
        File "<string>", line 2, in <module>
        File "<pip-setuptools-caller>", line 34, in <module>
        File "/tmp/pip-install-g954630y/libcom_1675273f43764efcbff80639b110a4a3/setup.py", line 114, in <module>
          setup(
        File "/root/miniconda3/envs/sapiens/lib/python3.10/site-packages/setuptools/__init__.py", line 117, in setup
          return distutils.core.setup(**attrs)
        File "/root/miniconda3/envs/sapiens/lib/python3.10/site-packages/setuptools/_distutils/core.py", line 183, in setup
          return run_commands(dist)
        File "/root/miniconda3/envs/sapiens/lib/python3.10/site-packages/setuptools/_distutils/core.py", line 199, in run_commands
          dist.run_commands()
        File "/root/miniconda3/envs/sapiens/lib/python3.10/site-packages/setuptools/_distutils/dist.py", line 954, in run_commands
          self.run_command(cmd)
        File "/root/miniconda3/envs/sapiens/lib/python3.10/site-packages/setuptools/dist.py", line 950, in run_command
          super().run_command(command)
        File "/root/miniconda3/envs/sapiens/lib/python3.10/site-packages/setuptools/_distutils/dist.py", line 973, in run_command
          cmd_obj.run()
        File "/root/miniconda3/envs/sapiens/lib/python3.10/site-packages/setuptools/command/bdist_wheel.py", line 398, in run
          self.run_command("build")
        File "/root/miniconda3/envs/sapiens/lib/python3.10/site-packages/setuptools/_distutils/cmd.py", line 316, in run_command
          self.distribution.run_command(command)
        File "/root/miniconda3/envs/sapiens/lib/python3.10/site-packages/setuptools/dist.py", line 950, in run_command
          super().run_command(command)
        File "/root/miniconda3/envs/sapiens/lib/python3.10/site-packages/setuptools/_distutils/dist.py", line 973, in run_command
          cmd_obj.run()
        File "/root/miniconda3/envs/sapiens/lib/python3.10/site-packages/setuptools/_distutils/command/build.py", line 135, in run
          self.run_command(cmd_name)
        File "/root/miniconda3/envs/sapiens/lib/python3.10/site-packages/setuptools/_distutils/cmd.py", line 316, in run_command
          self.distribution.run_command(command)
        File "/root/miniconda3/envs/sapiens/lib/python3.10/site-packages/setuptools/dist.py", line 950, in run_command
          super().run_command(command)
        File "/root/miniconda3/envs/sapiens/lib/python3.10/site-packages/setuptools/_distutils/dist.py", line 973, in run_command
          cmd_obj.run()
        File "/root/miniconda3/envs/sapiens/lib/python3.10/site-packages/setuptools/command/build_ext.py", line 98, in run
          _build_ext.run(self)
        File "/root/miniconda3/envs/sapiens/lib/python3.10/site-packages/setuptools/_distutils/command/build_ext.py", line 359, in run
          self.build_extensions()
        File "/root/miniconda3/envs/sapiens/lib/python3.10/site-packages/torch/utils/cpp_extension.py", line 870, in build_extensions
          build_ext.build_extensions(self)
        File "/root/miniconda3/envs/sapiens/lib/python3.10/site-packages/setuptools/_distutils/command/build_ext.py", line 476, in build_extensions
          self._build_extensions_serial()
        File "/root/miniconda3/envs/sapiens/lib/python3.10/site-packages/setuptools/_distutils/command/build_ext.py", line 502, in _build_extensions_serial
          self.build_extension(ext)
        File "/root/miniconda3/envs/sapiens/lib/python3.10/site-packages/setuptools/command/build_ext.py", line 263, in build_extension
          _build_ext.build_extension(self, ext)
        File "/root/miniconda3/envs/sapiens/lib/python3.10/site-packages/Cython/Distutils/build_ext.py", line 135, in build_extension
          super(build_ext, self).build_extension(ext)
        File "/root/miniconda3/envs/sapiens/lib/python3.10/site-packages/setuptools/_distutils/command/build_ext.py", line 557, in build_extension
          objects = self.compiler.compile(
        File "/root/miniconda3/envs/sapiens/lib/python3.10/site-packages/torch/utils/cpp_extension.py", line 683, in unix_wrap_ninja_compile
          _write_ninja_file_and_compile_objects(
        File "/root/miniconda3/envs/sapiens/lib/python3.10/site-packages/torch/utils/cpp_extension.py", line 1783, in _write_ninja_file_and_compile_objects
          _run_ninja_build(
        File "/root/miniconda3/envs/sapiens/lib/python3.10/site-packages/torch/utils/cpp_extension.py", line 2123, in _run_ninja_build
          raise RuntimeError(message) from e
      RuntimeError: Error compiling objects for extension
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for libcom
  Running setup.py clean for libcom
Failed to build libcom
ERROR: ERROR: Failed to build installable wheels for some pyproject.toml based projects (libcom)

When installing sapiens library, there are some failures.

Installing collected packages: sapiens_cv
  DEPRECATION: Legacy editable install of sapiens_cv==1.0.0 from file:///root/autodl-tmp/sapiens/cv (setup.py develop) is deprecated. pip 25.0 will enforce this behaviour change. A possible replacement is to add a pyproject.toml or enable --use-pep517, and use setuptools >= 64. If the resulting installation is not behaving as expected, try using --config-settings editable_mode=compat. Please consult the setuptools documentation for more information. Discussion can be found at https://github.com/pypa/pip/issues/11457
  Running setup.py develop for sapiens_cv
    Running command python setup.py develop
    /root/autodl-tmp/sapiens/cv/setup.py:11: DeprecationWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html
      from pkg_resources import DistributionNotFound, get_distribution, parse_version
    running develop
    /root/miniconda3/envs/sapiens/lib/python3.10/site-packages/setuptools/command/develop.py:41: EasyInstallDeprecationWarning: easy_install command is deprecated.
    !!

            ********************************************************************************
            Please avoid running ``setup.py`` and ``easy_install``.
            Instead, use pypa/build, pypa/installer or other
            standards-based tools.

            See https://github.com/pypa/setuptools/issues/917 for details.
            ********************************************************************************

    !!
      easy_install.initialize_options(self)
    /root/miniconda3/envs/sapiens/lib/python3.10/site-packages/setuptools/_distutils/cmd.py:66: SetuptoolsDeprecationWarning: setup.py install is deprecated.
    !!

            ********************************************************************************
            Please avoid running ``setup.py`` directly.
            Instead, use pypa/build, pypa/installer or other
            standards-based tools.

            See https://blog.ganssle.io/articles/2021/10/setup-py-deprecated.html for details.
            ********************************************************************************

    !!
      self.initialize_options()
    running egg_info
    writing sapiens_cv.egg-info/PKG-INFO
    writing dependency_links to sapiens_cv.egg-info/dependency_links.txt
    writing requirements to sapiens_cv.egg-info/requires.txt
    writing top-level names to sapiens_cv.egg-info/top_level.txt
    reading manifest file 'sapiens_cv.egg-info/SOURCES.txt'
    reading manifest template 'MANIFEST.in'
    writing manifest file 'sapiens_cv.egg-info/SOURCES.txt'
    running build_ext
    /root/miniconda3/envs/sapiens/lib/python3.10/site-packages/torch/utils/cpp_extension.py:428: UserWarning: There are no g++ version bounds defined for CUDA version 12.1
      warnings.warn(f'There are no {compiler_name} version bounds defined for CUDA version {cuda_str_version}')
    building 'mmcv._ext' extension
    /root/miniconda3/envs/sapiens/lib/python3.10/site-packages/torch/utils/cpp_extension.py:1967: UserWarning: TORCH_CUDA_ARCH_LIST is not set, all archs for visible cards are included for compilation.
    If this is not desired, please set os.environ['TORCH_CUDA_ARCH_LIST'].
      warnings.warn(
    Emitting ninja build file /root/autodl-tmp/sapiens/cv/build/temp.linux-x86_64-cpython-310/build.ninja...
    Compiling objects...
    Using envvar MAX_JOBS (111) as the number of workers...
    [1/83] /usr/local/cuda/bin/nvcc --generate-dependencies-with-compile --dependency-output /root/autodl-tmp/sapiens/cv/build/temp.linux-x86_64-cpython-310/mmcv/ops/csrc/pytorch/cuda/focal_loss_cuda.o.d -DMMCV_WITH_CUDA -I/root/autodl-tmp/sapiens/cv/mmcv/ops/csrc/pytorch -I/root/autodl-tmp/sapiens/cv/mmcv/ops/csrc/common -I/root/autodl-tmp/sapiens/cv/mmcv/ops/csrc/common/cuda -I/root/miniconda3/envs/sapiens/lib/python3.10/site-packages/torch/include -I/root/miniconda3/envs/sapiens/lib/python3.10/site-packages/torch/include/torch/csrc/api/include -I/root/miniconda3/envs/sapiens/lib/python3.10/site-packages/torch/include/TH -I/root/miniconda3/envs/sapiens/lib/python3.10/site-packages/torch/include/THC -I/usr/local/cuda/include -I/root/miniconda3/envs/sapiens/include/python3.10 -c -c /root/autodl-tmp/sapiens/cv/mmcv/ops/csrc/pytorch/cuda/focal_loss_cuda.cu -o /root/autodl-tmp/sapiens/cv/build/temp.linux-x86_64-cpython-310/mmcv/ops/csrc/pytorch/cuda/focal_loss_cuda.o -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_BFLOAT16_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr --compiler-options ''"'"'-fPIC'"'"'' -std=c++17 -DTORCH_API_INCLUDE_EXTENSION_H '-DPYBIND11_COMPILER_TYPE="_gcc"' '-DPYBIND11_STDLIB="_libstdcpp"' '-DPYBIND11_BUILD_ABI="_cxxabi1011"' -DTORCH_EXTENSION_NAME=_ext -D_GLIBCXX_USE_CXX11_ABI=0 -gencode=arch=compute_86,code=compute_86 -gencode=arch=compute_86,code=sm_86
    FAILED: /root/autodl-tmp/sapiens/cv/build/temp.linux-x86_64-cpython-310/mmcv/ops/csrc/pytorch/cuda/focal_loss_cuda.o
    /usr/local/cuda/bin/nvcc --generate-dependencies-with-compile --dependency-output /root/autodl-tmp/sapiens/cv/build/temp.linux-x86_64-cpython-310/mmcv/ops/csrc/pytorch/cuda/focal_loss_cuda.o.d -DMMCV_WITH_CUDA -I/root/autodl-tmp/sapiens/cv/mmcv/ops/csrc/pytorch -I/root/autodl-tmp/sapiens/cv/mmcv/ops/csrc/common -I/root/autodl-tmp/sapiens/cv/mmcv/ops/csrc/common/cuda -I/root/miniconda3/envs/sapiens/lib/python3.10/site-packages/torch/include -I/root/miniconda3/envs/sapiens/lib/python3.10/site-packages/torch/include/torch/csrc/api/include -I/root/miniconda3/envs/sapiens/lib/python3.10/site-packages/torch/include/TH -I/root/miniconda3/envs/sapiens/lib/python3.10/site-packages/torch/include/THC -I/usr/local/cuda/include -I/root/miniconda3/envs/sapiens/include/python3.10 -c -c /root/autodl-tmp/sapiens/cv/mmcv/ops/csrc/pytorch/cuda/focal_loss_cuda.cu -o /root/autodl-tmp/sapiens/cv/build/temp.linux-x86_64-cpython-310/mmcv/ops/csrc/pytorch/cuda/focal_loss_cuda.o -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_BFLOAT16_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr --compiler-options ''"'"'-fPIC'"'"'' -std=c++17 -DTORCH_API_INCLUDE_EXTENSION_H '-DPYBIND11_COMPILER_TYPE="_gcc"' '-DPYBIND11_STDLIB="_libstdcpp"' '-DPYBIND11_BUILD_ABI="_cxxabi1011"' -DTORCH_EXTENSION_NAME=_ext -D_GLIBCXX_USE_CXX11_ABI=0 -gencode=arch=compute_86,code=compute_86 -gencode=arch=compute_86,code=sm_86
    Killed
rawalkhirodkar commented 1 week ago

@Transfor-Cat thank for you sharing the logs. Is there an issue with your cuda setup? Do the following commands work for you?

nvidia-smi  # Check CUDA availability
nvcc -V     # Check CUDA compiler version
Transfor-Cat commented 1 week ago

@rawalkhirodkar I have already checked if the CUDA working. However, it seems that there is nothing wrong. I guess that this is related to some libraries relying on specific versions, but the required versions were not explicitly specified during installation. For example，the installation of the libcom library seems to require certain runtime environments, and I cannot guarantee that they have been installed (https://github.com/bcmi/libcom/blob/main/docs/get_started.md).

rawalkhirodkar commented 1 week ago

@Transfor-Cat that might explain the libcom crash, but the sapiens lib are installed from source. They should not have any version incompatibility issue.

Can you share your OS and gpu specs?

rawalkhirodkar commented 1 day ago

@Transfor-Cat closing due to inactivity. Feel free to reopen.