Closed monarchwise closed 1 year ago
楼上师兄,问题解决了没有?我编译时也遇到相同的问题。 OS:ubuntu18.04 gcc/g++:version 9.4.0 (Ubuntu 9.4.0-1ubuntu1~18.04)
错误: Collecting bmtrain Using cached bmtrain-0.2.2.tar.gz (58 kB) Preparing metadata (setup.py) ... done Requirement already satisfied: numpy in /home/haitaiwork/llm/anaconda3/lib/python3.10/site-packages (from bmtrain) (1.23.5) Building wheels for collected packages: bmtrain Building wheel for bmtrain (setup.py) ... error error: subprocess-exited-with-error
× python setup.py bdist_wheel did not run successfully.
│ exit code: 1
╰─> [102 lines of output]
No CUDA runtime is found, using CUDA_HOME='/usr/local/cuda'
running bdist_wheel
running build
running build_py
creating build
creating build/lib.linux-x86_64-cpython-310
creating build/lib.linux-x86_64-cpython-310/bmtrain
copying bmtrain/debug.py -> build/lib.linux-x86_64-cpython-310/bmtrain
copying bmtrain/init.py -> build/lib.linux-x86_64-cpython-310/bmtrain
copying bmtrain/layer.py -> build/lib.linux-x86_64-cpython-310/bmtrain
copying bmtrain/global_var.py -> build/lib.linux-x86_64-cpython-310/bmtrain
copying bmtrain/utils.py -> build/lib.linux-x86_64-cpython-310/bmtrain
copying bmtrain/checkpointing.py -> build/lib.linux-x86_64-cpython-310/bmtrain
copying bmtrain/parameter.py -> build/lib.linux-x86_64-cpython-310/bmtrain
copying bmtrain/store.py -> build/lib.linux-x86_64-cpython-310/bmtrain
copying bmtrain/block_layer.py -> build/lib.linux-x86_64-cpython-310/bmtrain
copying bmtrain/param_init.py -> build/lib.linux-x86_64-cpython-310/bmtrain
copying bmtrain/pipe_layer.py -> build/lib.linux-x86_64-cpython-310/bmtrain
copying bmtrain/init.py -> build/lib.linux-x86_64-cpython-310/bmtrain
copying bmtrain/wrapper.py -> build/lib.linux-x86_64-cpython-310/bmtrain
copying bmtrain/synchronize.py -> build/lib.linux-x86_64-cpython-310/bmtrain
creating build/lib.linux-x86_64-cpython-310/bmtrain/nccl
copying bmtrain/nccl/init.py -> build/lib.linux-x86_64-cpython-310/bmtrain/nccl
copying bmtrain/nccl/enums.py -> build/lib.linux-x86_64-cpython-310/bmtrain/nccl
creating build/lib.linux-x86_64-cpython-310/bmtrain/optim
copying bmtrain/optim/init.py -> build/lib.linux-x86_64-cpython-310/bmtrain/optim
copying bmtrain/optim/adam.py -> build/lib.linux-x86_64-cpython-310/bmtrain/optim
copying bmtrain/optim/optim_manager.py -> build/lib.linux-x86_64-cpython-310/bmtrain/optim
copying bmtrain/optim/adam_offload.py -> build/lib.linux-x86_64-cpython-310/bmtrain/optim
creating build/lib.linux-x86_64-cpython-310/bmtrain/benchmark
copying bmtrain/benchmark/init.py -> build/lib.linux-x86_64-cpython-310/bmtrain/benchmark
copying bmtrain/benchmark/shape.py -> build/lib.linux-x86_64-cpython-310/bmtrain/benchmark
copying bmtrain/benchmark/all_gather.py -> build/lib.linux-x86_64-cpython-310/bmtrain/benchmark
copying bmtrain/benchmark/send_recv.py -> build/lib.linux-x86_64-cpython-310/bmtrain/benchmark
copying bmtrain/benchmark/utils.py -> build/lib.linux-x86_64-cpython-310/bmtrain/benchmark
copying bmtrain/benchmark/reduce_scatter.py -> build/lib.linux-x86_64-cpython-310/bmtrain/benchmark
creating build/lib.linux-x86_64-cpython-310/bmtrain/inspect
copying bmtrain/inspect/init.py -> build/lib.linux-x86_64-cpython-310/bmtrain/inspect
copying bmtrain/inspect/format.py -> build/lib.linux-x86_64-cpython-310/bmtrain/inspect
copying bmtrain/inspect/model.py -> build/lib.linux-x86_64-cpython-310/bmtrain/inspect
copying bmtrain/inspect/tensor.py -> build/lib.linux-x86_64-cpython-310/bmtrain/inspect
creating build/lib.linux-x86_64-cpython-310/bmtrain/loss
copying bmtrain/loss/init.py -> build/lib.linux-x86_64-cpython-310/bmtrain/loss
copying bmtrain/loss/cross_entropy.py -> build/lib.linux-x86_64-cpython-310/bmtrain/loss
creating build/lib.linux-x86_64-cpython-310/bmtrain/lr_scheduler
copying bmtrain/lr_scheduler/no_decay.py -> build/lib.linux-x86_64-cpython-310/bmtrain/lr_scheduler
copying bmtrain/lr_scheduler/exponential.py -> build/lib.linux-x86_64-cpython-310/bmtrain/lr_scheduler
copying bmtrain/lr_scheduler/init.py -> build/lib.linux-x86_64-cpython-310/bmtrain/lr_scheduler
copying bmtrain/lr_scheduler/noam.py -> build/lib.linux-x86_64-cpython-310/bmtrain/lr_scheduler
copying bmtrain/lr_scheduler/cosine.py -> build/lib.linux-x86_64-cpython-310/bmtrain/lr_scheduler
copying bmtrain/lr_scheduler/warmup.py -> build/lib.linux-x86_64-cpython-310/bmtrain/lr_scheduler
copying bmtrain/lr_scheduler/linear.py -> build/lib.linux-x86_64-cpython-310/bmtrain/lr_scheduler
creating build/lib.linux-x86_64-cpython-310/bmtrain/distributed
copying bmtrain/distributed/ops.py -> build/lib.linux-x86_64-cpython-310/bmtrain/distributed
copying bmtrain/distributed/init.py -> build/lib.linux-x86_64-cpython-310/bmtrain/distributed
running buildext
Traceback (most recent call last):
File "
note: This error originates from a subprocess, and is likely not a problem with pip. ERROR: Failed building wheel for bmtrain Running setup.py clean for bmtrain Failed to build bmtrain Installing collected packages: bmtrain Running setup.py install for bmtrain ... error error: subprocess-exited-with-error
× Running setup.py install for bmtrain did not run successfully.
│ exit code: 1
╰─> [106 lines of output]
No CUDA runtime is found, using CUDA_HOME='/usr/local/cuda'
running install
/home/haitaiwork/llm/anaconda3/lib/python3.10/site-packages/setuptools/command/install.py:34: SetuptoolsDeprecationWarning: setup.py install is deprecated. Use build and pip and other standards-based tools.
warnings.warn(
running build
running build_py
creating build
creating build/lib.linux-x86_64-cpython-310
creating build/lib.linux-x86_64-cpython-310/bmtrain
copying bmtrain/debug.py -> build/lib.linux-x86_64-cpython-310/bmtrain
copying bmtrain/init.py -> build/lib.linux-x86_64-cpython-310/bmtrain
copying bmtrain/layer.py -> build/lib.linux-x86_64-cpython-310/bmtrain
copying bmtrain/global_var.py -> build/lib.linux-x86_64-cpython-310/bmtrain
copying bmtrain/utils.py -> build/lib.linux-x86_64-cpython-310/bmtrain
copying bmtrain/checkpointing.py -> build/lib.linux-x86_64-cpython-310/bmtrain
copying bmtrain/parameter.py -> build/lib.linux-x86_64-cpython-310/bmtrain
copying bmtrain/store.py -> build/lib.linux-x86_64-cpython-310/bmtrain
copying bmtrain/block_layer.py -> build/lib.linux-x86_64-cpython-310/bmtrain
copying bmtrain/param_init.py -> build/lib.linux-x86_64-cpython-310/bmtrain
copying bmtrain/pipe_layer.py -> build/lib.linux-x86_64-cpython-310/bmtrain
copying bmtrain/init.py -> build/lib.linux-x86_64-cpython-310/bmtrain
copying bmtrain/wrapper.py -> build/lib.linux-x86_64-cpython-310/bmtrain
copying bmtrain/synchronize.py -> build/lib.linux-x86_64-cpython-310/bmtrain
creating build/lib.linux-x86_64-cpython-310/bmtrain/nccl
copying bmtrain/nccl/init.py -> build/lib.linux-x86_64-cpython-310/bmtrain/nccl
copying bmtrain/nccl/enums.py -> build/lib.linux-x86_64-cpython-310/bmtrain/nccl
creating build/lib.linux-x86_64-cpython-310/bmtrain/optim
copying bmtrain/optim/init.py -> build/lib.linux-x86_64-cpython-310/bmtrain/optim
copying bmtrain/optim/adam.py -> build/lib.linux-x86_64-cpython-310/bmtrain/optim
copying bmtrain/optim/optim_manager.py -> build/lib.linux-x86_64-cpython-310/bmtrain/optim
copying bmtrain/optim/adam_offload.py -> build/lib.linux-x86_64-cpython-310/bmtrain/optim
creating build/lib.linux-x86_64-cpython-310/bmtrain/benchmark
copying bmtrain/benchmark/init.py -> build/lib.linux-x86_64-cpython-310/bmtrain/benchmark
copying bmtrain/benchmark/shape.py -> build/lib.linux-x86_64-cpython-310/bmtrain/benchmark
copying bmtrain/benchmark/all_gather.py -> build/lib.linux-x86_64-cpython-310/bmtrain/benchmark
copying bmtrain/benchmark/send_recv.py -> build/lib.linux-x86_64-cpython-310/bmtrain/benchmark
copying bmtrain/benchmark/utils.py -> build/lib.linux-x86_64-cpython-310/bmtrain/benchmark
copying bmtrain/benchmark/reduce_scatter.py -> build/lib.linux-x86_64-cpython-310/bmtrain/benchmark
creating build/lib.linux-x86_64-cpython-310/bmtrain/inspect
copying bmtrain/inspect/init.py -> build/lib.linux-x86_64-cpython-310/bmtrain/inspect
copying bmtrain/inspect/format.py -> build/lib.linux-x86_64-cpython-310/bmtrain/inspect
copying bmtrain/inspect/model.py -> build/lib.linux-x86_64-cpython-310/bmtrain/inspect
copying bmtrain/inspect/tensor.py -> build/lib.linux-x86_64-cpython-310/bmtrain/inspect
creating build/lib.linux-x86_64-cpython-310/bmtrain/loss
copying bmtrain/loss/init.py -> build/lib.linux-x86_64-cpython-310/bmtrain/loss
copying bmtrain/loss/cross_entropy.py -> build/lib.linux-x86_64-cpython-310/bmtrain/loss
creating build/lib.linux-x86_64-cpython-310/bmtrain/lr_scheduler
copying bmtrain/lr_scheduler/no_decay.py -> build/lib.linux-x86_64-cpython-310/bmtrain/lr_scheduler
copying bmtrain/lr_scheduler/exponential.py -> build/lib.linux-x86_64-cpython-310/bmtrain/lr_scheduler
copying bmtrain/lr_scheduler/init.py -> build/lib.linux-x86_64-cpython-310/bmtrain/lr_scheduler
copying bmtrain/lr_scheduler/noam.py -> build/lib.linux-x86_64-cpython-310/bmtrain/lr_scheduler
copying bmtrain/lr_scheduler/cosine.py -> build/lib.linux-x86_64-cpython-310/bmtrain/lr_scheduler
copying bmtrain/lr_scheduler/warmup.py -> build/lib.linux-x86_64-cpython-310/bmtrain/lr_scheduler
copying bmtrain/lr_scheduler/linear.py -> build/lib.linux-x86_64-cpython-310/bmtrain/lr_scheduler
creating build/lib.linux-x86_64-cpython-310/bmtrain/distributed
copying bmtrain/distributed/ops.py -> build/lib.linux-x86_64-cpython-310/bmtrain/distributed
copying bmtrain/distributed/init.py -> build/lib.linux-x86_64-cpython-310/bmtrain/distributed
running buildext
Traceback (most recent call last):
File "
note: This error originates from a subprocess, and is likely not a problem with pip. error: legacy-install-failure
× Encountered error while trying to install package. ╰─> bmtrain
note: This is an issue with the package mentioned above, not pip. hint: See above for output from the failure.
See also #125. It seems that your Python env not working. See https://stackoverflow.com/questions/49060525/install-jupyterlab-in-pip3-throws-typeerror-expected-string-or-bytes-like-obje. Or you can try newer version of BMTrain. PR #115 fixed some torch/cuda related bug.
CUDA环境变量也配置没问题,但还是提示这个,我把cuda版本从12.1降到11.8都是同样的错误 python setup.py install No CUDA runtime is found, using CUDA_HOME='/usr/local/cuda' running install /root/anaconda3/envs/cpm-bee/lib/python3.8/site-packages/setuptools/_distutils/cmd.py:66: SetuptoolsDeprecationWarning: setup.py install is deprecated. !!
!! self.initialize_options() /root/anaconda3/envs/cpm-bee/lib/python3.8/site-packages/setuptools/_distutils/cmd.py:66: EasyInstallDeprecationWarning: easy_install command is deprecated. !!
!! self.initialize_options() running bdist_egg running egg_info writing bmtrain.egg-info/PKG-INFO writing dependency_links to bmtrain.egg-info/dependency_links.txt writing requirements to bmtrain.egg-info/requires.txt writing top-level names to bmtrain.egg-info/top_level.txt reading manifest file 'bmtrain.egg-info/SOURCES.txt' reading manifest template 'MANIFEST.in' adding license file 'LICENSE' writing manifest file 'bmtrain.egg-info/SOURCES.txt' installing library code to build/bdist.linux-x86_64/egg running install_lib running build_py running build_ext Traceback (most recent call last): File "setup.py", line 74, in
setup(
File "/root/anaconda3/envs/cpm-bee/lib/python3.8/site-packages/setuptools/init.py", line 107, in setup
return distutils.core.setup(**attrs)
File "/root/anaconda3/envs/cpm-bee/lib/python3.8/site-packages/setuptools/_distutils/core.py", line 185, in setup
return run_commands(dist)
File "/root/anaconda3/envs/cpm-bee/lib/python3.8/site-packages/setuptools/_distutils/core.py", line 201, in run_commands
dist.run_commands()
File "/root/anaconda3/envs/cpm-bee/lib/python3.8/site-packages/setuptools/_distutils/dist.py", line 969, in run_commands
self.run_command(cmd)
File "/root/anaconda3/envs/cpm-bee/lib/python3.8/site-packages/setuptools/dist.py", line 1244, in run_command
super().run_command(command)
File "/root/anaconda3/envs/cpm-bee/lib/python3.8/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
cmd_obj.run()
File "/root/anaconda3/envs/cpm-bee/lib/python3.8/site-packages/setuptools/command/install.py", line 80, in run
self.do_egg_install()
File "/root/anaconda3/envs/cpm-bee/lib/python3.8/site-packages/setuptools/command/install.py", line 129, in do_egg_install
self.run_command('bdist_egg')
File "/root/anaconda3/envs/cpm-bee/lib/python3.8/site-packages/setuptools/_distutils/cmd.py", line 318, in run_command
self.distribution.run_command(command)
File "/root/anaconda3/envs/cpm-bee/lib/python3.8/site-packages/setuptools/dist.py", line 1244, in run_command
super().run_command(command)
File "/root/anaconda3/envs/cpm-bee/lib/python3.8/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
cmd_obj.run()
File "/root/anaconda3/envs/cpm-bee/lib/python3.8/site-packages/setuptools/command/bdist_egg.py", line 164, in run
cmd = self.call_command('install_lib', warn_dir=0)
File "/root/anaconda3/envs/cpm-bee/lib/python3.8/site-packages/setuptools/command/bdist_egg.py", line 150, in call_command
self.run_command(cmdname)
File "/root/anaconda3/envs/cpm-bee/lib/python3.8/site-packages/setuptools/_distutils/cmd.py", line 318, in run_command
self.distribution.run_command(command)
File "/root/anaconda3/envs/cpm-bee/lib/python3.8/site-packages/setuptools/dist.py", line 1244, in run_command
super().run_command(command)
File "/root/anaconda3/envs/cpm-bee/lib/python3.8/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
cmd_obj.run()
File "/root/anaconda3/envs/cpm-bee/lib/python3.8/site-packages/setuptools/command/install_lib.py", line 11, in run
self.build()
File "/root/anaconda3/envs/cpm-bee/lib/python3.8/site-packages/setuptools/_distutils/command/install_lib.py", line 111, in build
self.run_command('build_ext')
File "/root/anaconda3/envs/cpm-bee/lib/python3.8/site-packages/setuptools/_distutils/cmd.py", line 318, in run_command
self.distribution.run_command(command)
File "/root/anaconda3/envs/cpm-bee/lib/python3.8/site-packages/setuptools/dist.py", line 1244, in run_command
super().run_command(command)
File "/root/anaconda3/envs/cpm-bee/lib/python3.8/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
cmd_obj.run()
File "/root/anaconda3/envs/cpm-bee/lib/python3.8/site-packages/setuptools/command/build_ext.py", line 84, in run
_build_ext.run(self)
File "/root/anaconda3/envs/cpm-bee/lib/python3.8/site-packages/setuptools/_distutils/command/build_ext.py", line 345, in run
self.build_extensions()
File "/root/anaconda3/envs/cpm-bee/lib/python3.8/site-packages/torch/utils/cpp_extension.py", line 499, in build_extensions
_check_cuda_version(compiler_name, compiler_version)
File "/root/anaconda3/envs/cpm-bee/lib/python3.8/site-packages/torch/utils/cpp_extension.py", line 382, in _check_cuda_version
torch_cuda_version = packaging.version.parse(torch.version.cuda)
File "/root/anaconda3/envs/cpm-bee/lib/python3.8/site-packages/pkg_resources/_vendor/packaging/version.py", line 52, in parse
return Version(version)
File "/root/anaconda3/envs/cpm-bee/lib/python3.8/site-packages/pkg_resources/_vendor/packaging/version.py", line 195, in init
match = self._regex.search(version)
TypeError: expected string or bytes-like object