search

TomHeaven / tensorflow-osx-build

Off-the-shelf python package of tensorflow with CUDA support for Mac OS.
142 stars 20 forks source link

unknown type name 'cusparseSpVecDescr_t' error when building https://github.com/TomHeaven/tensorflow/releases/tag/v2.3.0-macos #22

Closed llv22 closed 3 years ago

llv22 commented 3 years ago

@TomHeaven 想问一下，你下面这个issue是如何解决的 我的环境 macOS 10.13.6 Xcode 10.1 CUDA 10.1 cuDNN 7.6.5 我看你在这个帖子里也在讨论： https://github.com/TomHeaven/tensorflow-osx-build/issues/18 在https://github.com/tensorflow/tensorflow/issues/40260 里面提到用环境变量，我尝试过，但仍然报错

ERROR: /Users/llv23/Documents/05_machine_learning/dl_gpu_mac/crack_by_TomHeaven/tensorflow-2.3.0-macos/tensorflow/stream_executor/cuda/BUILD:457:1: C++ compilation of rule '//tensorflow/stream_executor/cuda:cusparse_stub' failed (Exit 1)
In file included from tensorflow/stream_executor/cuda/cusparse_stub.cc:59:
./tensorflow/stream_executor/cuda/cusparse_10_1.inc:7786:21: error: unknown type name 'cusparseSpVecDescr_t'
cusparseCreateSpVec(cusparseSpVecDescr_t *spVecDescr, int64_t size, int64_t nnz,
                    ^
./tensorflow/stream_executor/cuda/cusparse_10_1.inc:7790:7: error: unknown type name 'cusparseSpVecDescr_t'; did you mean 'cusparseSpMatDescr_t'?
      cusparseSpVecDescr_t *, int64_t, int64_t, void *, void *,
      ^~~~~~~~~~~~~~~~~~~~
      cusparseSpMatDescr_t
bazel-out/darwin-opt/bin/external/local_config_cuda/cuda/_virtual_includes/cuda_headers_virtual/third_party/gpus/cuda/include/cusparse.h:6964:36: note: 'cusparseSpMatDescr_t' declared here
typedef struct cusparseSpMatDescr* cusparseSpMatDescr_t;
                                   ^
In file included from tensorflow/stream_executor/cuda/cusparse_stub.cc:59:
./tensorflow/stream_executor/cuda/cusparse_10_1.inc:7799:22: error: unknown type name 'cusparseSpVecDescr_t'; did you mean 'cusparseSpMatDescr_t'?
cusparseDestroySpVec(cusparseSpVecDescr_t spVecDescr) {
                     ^~~~~~~~~~~~~~~~~~~~
                     cusparseSpMatDescr_t
bazel-out/darwin-opt/bin/external/local_config_cuda/cuda/_virtual_includes/cuda_headers_virtual/third_party/gpus/cuda/include/cusparse.h:6964:36: note: 'cusparseSpMatDescr_t' declared here
typedef struct cusparseSpMatDescr* cusparseSpMatDescr_t;
                                   ^
In file included from tensorflow/stream_executor/cuda/cusparse_stub.cc:59:
./tensorflow/stream_executor/cuda/cusparse_10_1.inc:7800:51: error: unknown type name 'cusparseSpVecDescr_t'; did you mean 'cusparseSpMatDescr_t'?
  using FuncPtr = cusparseStatus_t(CUSPARSEAPI *)(cusparseSpVecDescr_t);
                                                  ^~~~~~~~~~~~~~~~~~~~
                                                  cusparseSpMatDescr_t
bazel-out/darwin-opt/bin/external/local_config_cuda/cuda/_virtual_includes/cuda_headers_virtual/third_party/gpus/cuda/include/cusparse.h:6964:36: note: 'cusparseSpMatDescr_t' declared here
typedef struct cusparseSpMatDescr* cusparseSpMatDescr_t;
                                   ^
In file included from tensorflow/stream_executor/cuda/cusparse_stub.cc:59:
./tensorflow/stream_executor/cuda/cusparse_10_1.inc:7807:11: error: unknown type name 'cusparseSpVecDescr_t'; did you mean 'cusparseSpMatDescr_t'?
    const cusparseSpVecDescr_t spVecDescr, int64_t *size, int64_t *nnz,
          ^~~~~~~~~~~~~~~~~~~~
          cusparseSpMatDescr_t
bazel-out/darwin-opt/bin/external/local_config_cuda/cuda/_virtual_includes/cuda_headers_virtual/third_party/gpus/cuda/include/cusparse.h:6964:36: note: 'cusparseSpMatDescr_t' declared here
typedef struct cusparseSpMatDescr* cusparseSpMatDescr_t;
                                   ^
In file included from tensorflow/stream_executor/cuda/cusparse_stub.cc:59:
./tensorflow/stream_executor/cuda/cusparse_10_1.inc:7811:13: error: C++ requires a type specifier for all declarations
      const cusparseSpVecDescr_t, int64_t *, int64_t *, void **, void **,
      ~~~~~ ^
./tensorflow/stream_executor/cuda/cusparse_10_1.inc:7815:19: error: cannot initialize a parameter of type 'int' with an lvalue of type 'const cusparseSpMatDescr_t' (aka 'cusparseSpMatDescr *const')
  return func_ptr(spVecDescr, size, nnz, indices, values, idxType, idxBase,
                  ^~~~~~~~~~
./tensorflow/stream_executor/cuda/cusparse_10_1.inc:7820:11: error: unknown type name 'cusparseSpVecDescr_t'; did you mean 'cusparseSpMatDescr_t'?
    const cusparseSpVecDescr_t spVecDescr, cusparseIndexBase_t *idxBase) {
          ^~~~~~~~~~~~~~~~~~~~
          cusparseSpMatDescr_t
bazel-out/darwin-opt/bin/external/local_config_cuda/cuda/_virtual_includes/cuda_headers_virtual/third_party/gpus/cuda/include/cusparse.h:6964:36: note: 'cusparseSpMatDescr_t' declared here
typedef struct cusparseSpMatDescr* cusparseSpMatDescr_t;
                                   ^
In file included from tensorflow/stream_executor/cuda/cusparse_stub.cc:59:
./tensorflow/stream_executor/cuda/cusparse_10_1.inc:7821:57: error: C++ requires a type specifier for all declarations
  using FuncPtr = cusparseStatus_t(CUSPARSEAPI *)(const cusparseSpVecDescr_t,
                                                  ~~~~~ ^
./tensorflow/stream_executor/cuda/cusparse_10_1.inc:7825:19: error: cannot initialize a parameter of type 'int' with an lvalue of type 'const cusparseSpMatDescr_t' (aka 'cusparseSpMatDescr *const')
  return func_ptr(spVecDescr, idxBase);
                  ^~~~~~~~~~
./tensorflow/stream_executor/cuda/cusparse_10_1.inc:7829:30: error: unknown type name 'cusparseSpVecDescr_t'; did you mean 'cusparseSpMatDescr_t'?
cusparseSpVecGetValues(const cusparseSpVecDescr_t spVecDescr, void **values) {
                             ^~~~~~~~~~~~~~~~~~~~
                             cusparseSpMatDescr_t
bazel-out/darwin-opt/bin/external/local_config_cuda/cuda/_virtual_includes/cuda_headers_virtual/third_party/gpus/cuda/include/cusparse.h:6964:36: note: 'cusparseSpMatDescr_t' declared here
typedef struct cusparseSpMatDescr* cusparseSpMatDescr_t;
                                   ^
In file included from tensorflow/stream_executor/cuda/cusparse_stub.cc:59:
./tensorflow/stream_executor/cuda/cusparse_10_1.inc:7831:45: error: C++ requires a type specifier for all declarations
      cusparseStatus_t(CUSPARSEAPI *)(const cusparseSpVecDescr_t, void **);
                                      ~~~~~ ^
./tensorflow/stream_executor/cuda/cusparse_10_1.inc:7834:19: error: cannot initialize a parameter of type 'int' with an lvalue of type 'const cusparseSpMatDescr_t' (aka 'cusparseSpMatDescr *const')
  return func_ptr(spVecDescr, values);
                  ^~~~~~~~~~
./tensorflow/stream_executor/cuda/cusparse_10_1.inc:7838:24: error: unknown type name 'cusparseSpVecDescr_t'; did you mean 'cusparseSpMatDescr_t'?
cusparseSpVecSetValues(cusparseSpVecDescr_t spVecDescr, void *values) {
                       ^~~~~~~~~~~~~~~~~~~~
                       cusparseSpMatDescr_t
bazel-out/darwin-opt/bin/external/local_config_cuda/cuda/_virtual_includes/cuda_headers_virtual/third_party/gpus/cuda/include/cusparse.h:6964:36: note: 'cusparseSpMatDescr_t' declared here
typedef struct cusparseSpMatDescr* cusparseSpMatDescr_t;
                                   ^
In file included from tensorflow/stream_executor/cuda/cusparse_stub.cc:59:
./tensorflow/stream_executor/cuda/cusparse_10_1.inc:7839:51: error: C++ requires a type specifier for all declarations
  using FuncPtr = cusparseStatus_t(CUSPARSEAPI *)(cusparseSpVecDescr_t, void *);
                                                  ^
./tensorflow/stream_executor/cuda/cusparse_10_1.inc:7842:19: error: cannot initialize a parameter of type 'int' with an lvalue of type 'cusparseSpMatDescr_t' (aka 'cusparseSpMatDescr *')
  return func_ptr(spVecDescr, values);
                  ^~~~~~~~~~
./tensorflow/stream_executor/cuda/cusparse_10_1.inc:7846:21: error: unknown type name 'cusparseDnVecDescr_t'
cusparseCreateDnVec(cusparseDnVecDescr_t *dnVecDescr, int64_t size,
                    ^
./tensorflow/stream_executor/cuda/cusparse_10_1.inc:7849:7: error: unknown type name 'cusparseDnVecDescr_t'; did you mean 'cusparseDnMatDescr_t'?
      cusparseDnVecDescr_t *, int64_t, void *, cudaDataType);
      ^~~~~~~~~~~~~~~~~~~~
      cusparseDnMatDescr_t
bazel-out/darwin-opt/bin/external/local_config_cuda/cuda/_virtual_includes/cuda_headers_virtual/third_party/gpus/cuda/include/cusparse.h:6965:36: note: 'cusparseDnMatDescr_t' declared here
typedef struct cusparseDnMatDescr* cusparseDnMatDescr_t;
                                   ^
In file included from tensorflow/stream_executor/cuda/cusparse_stub.cc:59:
./tensorflow/stream_executor/cuda/cusparse_10_1.inc:7856:22: error: unknown type name 'cusparseDnVecDescr_t'; did you mean 'cusparseDnMatDescr_t'?
cusparseDestroyDnVec(cusparseDnVecDescr_t dnVecDescr) {
                     ^~~~~~~~~~~~~~~~~~~~
                     cusparseDnMatDescr_t
bazel-out/darwin-opt/bin/external/local_config_cuda/cuda/_virtual_includes/cuda_headers_virtual/third_party/gpus/cuda/include/cusparse.h:6965:36: note: 'cusparseDnMatDescr_t' declared here
typedef struct cusparseDnMatDescr* cusparseDnMatDescr_t;
                                   ^
fatal error: too many errors emitted, stopping now [-ferror-limit=]

我的环境变量如下：

(base) Orlando:~ llv23$ env
CONDA3_HOME=/Users/llv23/miniconda3
SPARK_HOME=/usr/local/Cellar/spark
rvm_bin_path=/Users/llv23/.rvm/bin
TERM_PROGRAM=Apple_Terminal
DYLD_FALLBACK_LIBRARY_PATH=/usr/local/cuda/lib:/Developer/NVIDIA/CUDA-10.1/lib:/usr/local/cuda/extras/CUPTI/lib:/usr/local/opt/boost-python3/lib:/usr/local/lib
MONOGACPREFIX=/usr/local
GEM_HOME=/Users/llv23/.rvm/gems/ruby-2.2.1
SHELL=/bin/bash
TERM=xterm-256color
HADOOP_HOME=/Users/llv23/Documents/00_devbase/04_hadoop/hadoop-2.7.3-build
PROTO_HOME=/usr/local/opt/protobuf@2.5
IRBRC=/Users/llv23/.rvm/rubies/ruby-2.2.1/.irbrc
TMPDIR=/var/folders/p8/91_v9_9d12q9wmlydb406rbr0000gn/T/
CONDA_SHLVL=1
GRADLE_HOME=/usr/local/Cellar/gradle/4.4.1
Apple_PubSub_Socket_Render=/private/tmp/com.apple.launchd.XZkcTSZp6r/Render
CONDA_PROMPT_MODIFIER=(base) 
TERM_PROGRAM_VERSION=404
OPENSSL_INCLUDE_DIR=/usr/local/opt/openssl/include
KAGGLE_CLI=~/.local/bin
MONGODB_HOME=/Users/llv23/Documents/orlando_innovation/homekits_sol/mongodb
MY_RUBY_HOME=/Users/llv23/.rvm/rubies/ruby-2.2.1
TERM_SESSION_ID=E38C7210-4C8B-412B-B9F9-F427A7101663
LC_ALL=en_US.UTF-8
OPENSSL_HOME=/usr/local/opt/openssl
CAFFE_HOME=/Users/llv23/Documents/05_machine_learning/dl_gpu_mac/caffe
CUDA_HOME=/usr/local/cuda
USER=llv23
LD_LIBRARY_PATH=/usr/local/cuda/lib:/Developer/NVIDIA/CUDA-10.1/lib:/usr/local/nccl/lib:/usr/local/cuda/extras/CUPTI/lib:/usr/local/opt/boost-python3/lib:/usr/local/opt/open-mpi/lib:/Users/llv23/miniconda3/lib:/usr/local/opt/open-mpi/lib:/usr/local/lib
OCTAVE_BIN=/Applications/Octave.app/Contents/Resources/usr/bin
_system_type=Darwin
CONDA_EXE=/Users/llv23/opt/miniconda3/bin/conda
rvm_path=/Users/llv23/.rvm
MONO_HOME=/usr/local/Cellar/mono/5.4.1.6
PROTOBUF_HOME=/usr/local/protobuf
SSH_AUTH_SOCK=/private/tmp/com.apple.launchd.u5DZAObZFN/Listeners
_CE_CONDA=
rvm_prefix=/Users/llv23
PATH=/Library/Frameworks/Python.framework/Versions/3.7/bin:/Library/Frameworks/Python.framework/Versions/3.6/bin:/Users/llv23/opt/miniconda3/bin:/Users/llv23/opt/miniconda3/condabin:~/.local/bin:/usr/local/cuda/lib:/Developer/NVIDIA/CUDA-10.1/lib:/usr/local/nccl/lib:/usr/local/cuda/extras/CUPTI/lib:/usr/local/opt/boost-python3/lib:/usr/local/opt/open-mpi/lib:/usr/local/Cellar/spark/bin:/usr/local/sbin:/usr/local/cuda/bin:/Developer/NVIDIA/CUDA-10.1/bin:/opt/local/bin:/opt/local/sbin:/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin:/Users/llv23/miniconda3/bin:/usr/local/opt/protobuf@2.5/bin:/usr/local/opt/protobuf@2.5/bin:/usr/local/opt/openssl/bin:/usr/local/opt/node@6/bin:/usr/local/share/dotnet:/Users/llv23/.rvm/gems/ruby-2.2.1/bin:/Users/llv23/.rvm/gems/ruby-2.2.1@global/bin:/Users/llv23/.rvm/rubies/ruby-2.2.1/bin:/Users/llv23/.dnx/runtimes/dnx-mono.1.0.0-beta4/bin:/usr/local/protobuf/bin:/usr/local/Cellar/scala/2.12.4/bin:/usr/local/bin/vmware/Library:/Users/llv23/Documents/orlando_innovation/homekits_sol/mongodb/bin:/Library/Frameworks/GDAL.framework/Programs:/Users/llv23/Documents/02_apple_programming/06_classdump:/Applications/VirtualBox.app/Contents:/Users/llv23/Documents/04_linuxc/07_redis/redis-2.6.13/src:/Users/llv23/npm-global/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/share/dotnet:/opt/X11/bin:/Library/Frameworks/Mono.framework/Versions/Current/Commands:/Users/llv23/.rvm/bin
NPM_CONFIG_PREFIX=/Users/llv23/npm-global/
CONDA_PREFIX=/Users/llv23/opt/miniconda3
PWD=/Users/llv23
JAVA_HOME=/Library/Java/JavaVirtualMachines/jdk-11.0.4.jdk/Contents/Home
MXNET_GLUON_REPO=https://apache-mxnet.s3.cn-north-1.amazonaws.com.cn/
LANG=en_US.UTF-8
LUA_CPATH=/Users/llv23/Documents/04_linuxc/07_redis/01_lua/scripts/uuid/uuid.so
_system_arch=x86_64
GRE340=/Users/llv23/Library/Mobile Documents/com~apple~CloudDocs/grefilter
XPC_FLAGS=0x0
SWIFT_HOME=/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain
_system_version=10.13
_CE_M=
XPC_SERVICE_NAME=0
FLINK_HOME=/usr/local/Cellar/apache-flink/1.3.2/libexec
DOTNET_HOME=/usr/local/share/dotnet
rvm_version=1.29.3 (latest)
OPENSSL_ROOT_DIR=/usr/local/opt/openssl
SHLVL=1
HOME=/Users/llv23
OPENMPI_HOME=/usr/local/opt/open-mpi
DYLD_LIBRARY_PATH=/usr/local/cuda/lib:/Developer/NVIDIA/CUDA-10.1/lib:/usr/local/nccl/lib:/usr/local/cuda/extras/CUPTI/lib:/usr/local/opt/boost-python3/lib:/usr/local/opt/open-mpi/lib
CONDA_PYTHON_EXE=/Users/llv23/opt/miniconda3/bin/python
PYTHONPATH=/Users/llv23/Documents/05_machine_learning/dl_gpu_mac/caffe/python:
LOGNAME=llv23
GEM_PATH=/Users/llv23/.rvm/gems/ruby-2.2.1:/Users/llv23/.rvm/gems/ruby-2.2.1@global
LC_CTYPE=UTF-8
CONDA_DEFAULT_ENV=base
SCALA_HOME=/usr/local/opt/scala
VMWARE_HOME=/usr/local/bin/vmware
DISPLAY=/private/tmp/com.apple.launchd.bcvccljMRS/org.macosforge.xquartz:0
RUBY_VERSION=ruby-2.2.1
NODE_HOME=/usr/local/opt/node@6
_system_name=OSX
M3_HOME=/usr/local/Cellar/maven/3.3.9
NPM_HOME=/Users/llv23/npm-global/
_=/usr/bin/env

我比较了和你的env差别，怀疑是不是cuda的include缺失；我加了试验一下，让机器跑个晚上，看明天是不是还有这个错

# perhaps missing header
export SYS_C_INCLUDE="/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.14.sdk/usr/include"
export C_INCLUDE_PATH=/usr/local/include:/usr/local/cuda/include:${SYS_C_INCLUDE}
export CXX_INCLUDE_PATH=/usr/local/include:/usr/local/cuda/include:${SYS_C_INCLUDE}

刚刚find了一下，结果发现确实在/usr/local/cuda/include/cusparse.h里面，soft link到/Developer/NVIDIA/CUDA-10.1/include/cusparse.h ，希望明早起来这个错误消失吧，编译一次时间巨长 还是没有找到，感觉在include bazel-out/host/bin/external/local_config_cuda/cuda/_virtual_includes/cuda_headers_virtual/third_party/gpus/cuda/include/cusparse.h的时候，有问题

大兄弟，我发现我的CUDA10.1好像不是update2，NVIDA应该是在10.1update2之后才加入的cusparseSpVecDescr_t。。。哎，hopefully sail through

llv22 commented 3 years ago

搞定，关帖