search

bytedance / byteps

A high performance and generic framework for distributed DNN training
Other
3.63k stars 490 forks source link

安装报错 #433

Open llplay opened 2 years ago

llplay commented 2 years ago

安装报错，是什么依赖安装的不对吗，辛苦看下 python 3.7.5 tensorflow 2.5.0

` x86_64-linux-gnu-gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -g -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -DEIGEN_MPL2_ONLY=1 -I3rdparty/ps-lite/include -I/usr/local/nccl/include -I/usr/include/python3.7m -c byteps/common/compressor/impl/dithering.cc -o build/temp.linux-x86_64-3.7/byteps/common/compressor/impl/dithering.o -std=c++11 -fPIC -Ofast -Wall -fopenmp -march=native -mno-avx512f -D_GLIBCXX_USE_CXX11_ABI=0 -DBYTEPS_BUILDING_SERVER In file included from byteps/common/compressor/impl/../compressor.h:23:0, from byteps/common/compressor/impl/../compressor_registry.h:19, from byteps/common/compressor/impl/dithering.cc:19: byteps/common/compressor/impl/dithering.cc: In member function ‘virtual byteps::common::compressor::tensor_t byteps::common::compressor::DitheringCompressor::Compress(byteps::common::compressor::tensor_t)’: byteps/common/compressor/impl/../common.h:48:42: error: ISO C++ forbids declaration of ‘type name’ with no type [-fpermissive] reinterpret_cast<const half_t>(src), \ ^ byteps/common/compressor/impl/dithering.cc:119:3: note: in expansion of macro ‘COMPRESS_IMPL_SWITCH’ COMPRESS_IMPL_SWITCH(grad.dtype, CompressImpl, _buf.get(), grad.data, ^~~~~~~~ byteps/common/compressor/impl/../common.h:48:42: error: expected ‘>’ before ‘half_t’ reinterpret_cast<const half_t>(src), \ ^ byteps/common/compressor/impl/dithering.cc:119:3: note: in expansion of macro ‘COMPRESS_IMPL_SWITCH’ COMPRESS_IMPL_SWITCH(grad.dtype, CompressImpl, _buf.get(), grad.data, ^~~~~~~~ byteps/common/compressor/impl/../common.h:48:42: error: expected ‘(’ before ‘half_t’ reinterpret_cast<const half_t>(src), \ ^ byteps/common/compressor/impl/dithering.cc:119:3: note: in expansion of macro ‘COMPRESS_IMPL_SWITCH’ COMPRESS_IMPL_SWITCH(grad.dtype, CompressImpl, _buf.get(), grad.data, ^~~~~~~~ byteps/common/compressor/impl/../common.h:48:42: error: ‘half_t’ was not declared in this scope reinterpret_cast<const half_t>(src), \ ^ byteps/common/compressor/impl/dithering.cc:119:3: note: in expansion of macro ‘COMPRESS_IMPL_SWITCH’ COMPRESS_IMPL_SWITCH(grad.dtype, CompressImpl, _buf.get(), grad.data, ^~~~~~~~ byteps/common/compressor/impl/../common.h:48:42: note: suggested alternative: ‘off_t’ reinterpret_cast<const half_t>(src), \ ^ byteps/common/compressor/impl/dithering.cc:119:3: note: in expansion of macro ‘COMPRESS_IMPL_SWITCH’ COMPRESS_IMPL_SWITCH(grad.dtype, CompressImpl, _buf.get(), grad.data, ^~~~~~~~ byteps/common/compressor/impl/../common.h:48:49: error: expected primary-expression before ‘>’ token reinterpret_cast<const half_t>(src), \ ^ byteps/common/compressor/impl/dithering.cc:119:3: note: in expansion of macro ‘COMPRESS_IMPL_SWITCH’ COMPRESS_IMPL_SWITCH(grad.dtype, CompressImpl, _buf.get(), grad.data, ^~~~~~~~ byteps/common/compressor/impl/dithering.cc: In member function ‘virtual byteps::common::compressor::tensor_t byteps::common::compressor::DitheringCompressor::Decompress(byteps::common::compressor::tensor_t)’: byteps/common/compressor/impl/../common.h:64:36: error: ‘half_t’ does not name a type; did you mean ‘off_t’? return func(reinterpret_cast<half_t>(dst), \ ^ byteps/common/compressor/impl/dithering.cc:170:3: note: in expansion of macro ‘DECOMPRESS_IMPL_SWITCH’ DECOMPRESS_IMPL_SWITCH(_dtype, DecompressImpl, dst, compressed.data, ^~~~~~ byteps/common/compressor/impl/../common.h:64:42: error: expected ‘>’ before ‘’ token return func(reinterpret_cast<half_t>(dst), \ ^ byteps/common/compressor/impl/dithering.cc:170:3: note: in expansion of macro ‘DECOMPRESS_IMPL_SWITCH’ DECOMPRESS_IMPL_SWITCH(_dtype, DecompressImpl, dst, compressed.data, ^~~~~~ byteps/common/compressor/impl/../common.h:64:42: error: expected ‘(’ before ‘’ token return func(reinterpret_cast<half_t>(dst), \ ^ byteps/common/compressor/impl/dithering.cc:170:3: note: in expansion of macro ‘DECOMPRESS_IMPL_SWITCH’ DECOMPRESS_IMPL_SWITCH(_dtype, DecompressImpl, dst, compressed.data, ^~~~~~ byteps/common/compressor/impl/../common.h:64:43: error: expected primary-expression before ‘>’ token return func(reinterpret_cast<half_t>(dst), \ ^ byteps/common/compressor/impl/dithering.cc:170:3: note: in expansion of macro ‘DECOMPRESS_IMPL_SWITCH’ DECOMPRESS_IMPL_SWITCH(_dtype, DecompressImpl, dst, compressed.data, ^~~~~~ byteps/common/compressor/impl/dithering.cc: In member function ‘virtual void byteps::common::compressor::DitheringCompressor::FastUpdateError(byteps::common::compressor::tensor_t, byteps::common::compressor::tensor_t, byteps::common::compressor::tensor_t)’: byteps/common/compressor/impl/../common.h:80:36: error: ‘half_t’ does not name a type; did you mean ‘off_t’? return func(reinterpret_cast<half_t>(dst), \ ^ byteps/common/compressor/impl/dithering.cc:212:3: note: in expansion of macro ‘FAST_UPDATE_ERROR_IMPL_SWITCH’ FAST_UPDATE_ERROR_IMPL_SWITCH(_dtype, FastUpdateErrorImpl, error.data, ^~~~~~~~~ byteps/common/compressor/impl/../common.h:80:42: error: expected ‘>’ before ‘’ token return func(reinterpret_cast<half_t>(dst), \ ^ byteps/common/compressor/impl/dithering.cc:212:3: note: in expansion of macro ‘FAST_UPDATE_ERROR_IMPL_SWITCH’ FAST_UPDATE_ERROR_IMPL_SWITCH(_dtype, FastUpdateErrorImpl, error.data, ^~~~~~~~~ byteps/common/compressor/impl/../common.h:80:42: error: expected ‘(’ before ‘’ token return func(reinterpret_cast<half_t>(dst), \ ^ byteps/common/compressor/impl/dithering.cc:212:3: note: in expansion of macro ‘FAST_UPDATE_ERROR_IMPL_SWITCH’ FAST_UPDATE_ERROR_IMPL_SWITCH(_dtype, FastUpdateErrorImpl, error.data, ^~~~~~~~~ byteps/common/compressor/impl/../common.h:80:43: error: expected primary-expression before ‘>’ token return func(reinterpret_cast<half_t>(dst), \ ^ byteps/common/compressor/impl/dithering.cc:212:3: note: in expansion of macro ‘FAST_UPDATE_ERROR_IMPL_SWITCH’ FAST_UPDATE_ERROR_IMPL_SWITCH(_dtype, FastUpdateErrorImpl, error.data, ^~~~~~~~~ byteps/common/compressor/impl/../common.h:81:36: error: ‘half_t’ does not name a type; did you mean ‘off_t’? reinterpret_cast<half_t>(src1), \ ^ byteps/common/compressor/impl/dithering.cc:212:3: note: in expansion of macro ‘FAST_UPDATE_ERROR_IMPL_SWITCH’ FAST_UPDATE_ERROR_IMPL_SWITCH(_dtype, FastUpdateErrorImpl, error.data, ^~~~~~~~~ byteps/common/compressor/impl/../common.h:81:42: error: expected ‘>’ before ‘’ token reinterpret_cast<half_t>(src1), \ ^ byteps/common/compressor/impl/dithering.cc:212:3: note: in expansion of macro ‘FAST_UPDATE_ERROR_IMPL_SWITCH’ FAST_UPDATE_ERROR_IMPL_SWITCH(_dtype, FastUpdateErrorImpl, error.data, ^~~~~~~~~ byteps/common/compressor/impl/../common.h:81:42: error: expected ‘(’ before ‘’ token reinterpret_cast<half_t>(src1), \ ^ byteps/common/compressor/impl/dithering.cc:212:3: note: in expansion of macro ‘FAST_UPDATE_ERROR_IMPL_SWITCH’ FAST_UPDATE_ERROR_IMPL_SWITCH(_dtype, FastUpdateErrorImpl, error.data, ^~~~~~~~~ byteps/common/compressor/impl/../common.h:81:43: error: expected primary-expression before ‘>’ token reinterpret_cast<half_t>(src1), \ ^ byteps/common/compressor/impl/dithering.cc:212:3: note: in expansion of macro ‘FAST_UPDATE_ERROR_IMPL_SWITCH’ FAST_UPDATE_ERROR_IMPL_SWITCH(_dtype, FastUpdateErrorImpl, error.data, ^~~~~~~~~ byteps/common/compressor/impl/../common.h:82:76: error: expected ‘)’ before ‘;’ token reinterpret_cast<const uint16_t>(src2), compressed_size); \ ^ byteps/common/compressor/impl/dithering.cc:212:3: note: in expansion of macro ‘FAST_UPDATE_ERROR_IMPL_SWITCH’ FAST_UPDATE_ERROR_IMPL_SWITCH(_dtype, FastUpdateErrorImpl, error.data, ^~~~~~~~~ byteps/common/compressor/impl/../common.h:82:76: error: return-statement with a value, in function returning 'void' [-fpermissive] reinterpret_cast<const uint16_t>(src2), compressed_size); \ ^ byteps/common/compressor/impl/dithering.cc:212:3: note: in expansion of macro ‘FAST_UPDATE_ERROR_IMPL_SWITCH’ FAST_UPDATE_ERROR_IMPL_SWITCH(_dtype, FastUpdateErrorImpl, error.data, ^~~~~~~~~ error: An ERROR occured while building the server module.

Traceback (most recent call last):
  File "/usr/lib/python3.7/distutils/unixccompiler.py", line 118, in _compile
    extra_postargs)
  File "/usr/lib/python3.7/distutils/ccompiler.py", line 910, in spawn
    spawn(cmd, dry_run=self.dry_run)
  File "/usr/lib/python3.7/distutils/spawn.py", line 36, in spawn
    _spawn_posix(cmd, search_path, dry_run=dry_run)
  File "/usr/lib/python3.7/distutils/spawn.py", line 159, in _spawn_posix
    % (cmd, exit_status))
distutils.errors.DistutilsExecError: command 'x86_64-linux-gnu-gcc' failed with exit status 1

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/tmp/pip-install-gg9uk_gv/byteps_d2daed5e88db4e3dbcc5a12776b83bdf/setup.py", line 944, in build_extensions
    build_server(self, options)
  File "/tmp/pip-install-gg9uk_gv/byteps_d2daed5e88db4e3dbcc5a12776b83bdf/setup.py", line 337, in build_server
    build_ext.build_extension(server_lib)
  File "/usr/local/lib/python3.7/dist-packages/setuptools/command/build_ext.py", line 196, in build_extension
    _build_ext.build_extension(self, ext)
  File "/usr/lib/python3.7/distutils/command/build_ext.py", line 534, in build_extension
    depends=ext.depends)
  File "/usr/lib/python3.7/distutils/ccompiler.py", line 574, in compile
    self._compile(obj, src, ext, cc_args, extra_postargs, pp_opts)
  File "/usr/lib/python3.7/distutils/unixccompiler.py", line 120, in _compile
    raise CompileError(msg)
distutils.errors.CompileError: command 'x86_64-linux-gnu-gcc' failed with exit status 1

----------------------------------------

ERROR: Command errored out with exit status 1: /usr/bin/python -u -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-gg9uk_gv/byteps_d2daed5e88db4e3dbcc5a12776b83bdf/setup.py'"'"'; file='"'"'/tmp/pip-install-gg9uk_gv/bytepsd2daed5e88db4e3dbcc5a12776b83bdf/setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(file) if os.path.exists(file) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, file, '"'"'exec'"'"'))' install --record /tmp/pip-record-6671etx/install-record.txt --single-version-externally-managed --compile --install-headers /usr/local/include/python3.7/byteps Check the logs for full command output. `

pleasantrabbit commented 2 years ago

@llplay sorry for the delay. what's the installation command you used? the following command works fine in my environment:

$ python3 -V
Python 3.7.3

$ pip3 show tensorflow
Name: tensorflow
Version: 2.5.0

$ python3 setup.py sdist
$ pip3 install dist/byteps-0.2.5.tar.gz