search

ctongfei / nexus

Experimental tensor-typed deep learning
https://tongfei.me/nexus/
MIT License
256 stars 15 forks source link

Issue in installing `Torch` (MacOS) #26

Open danyaljj opened 5 years ago

danyaljj commented 5 years ago

After trying to build it, here is what I get: 

> ./build.sh 
PyTorch 1.0.0 at /Users/daniel/miniconda3/lib/python3.7/site-packages/torch detected
Operating system is darwin17
Copying include files...
cp: /Users/daniel/miniconda3/lib/python3.7/site-packages/torch/lib/include/THC: No such file or directory

Taking a closer look into the include folder, here is what I see: 

> ls -lha /Users/daniel/miniconda3/lib/python3.7/site-packages/torch/lib/include/ 
total 0
drwxr-xr-x    9 daniel  staff   288B Dec 30 15:48 .
drwxr-xr-x   20 daniel  staff   640B Dec 30 15:48 ..
drwxr-xr-x  143 daniel  staff   4.5K Dec 30 15:48 ATen
drwxr-xr-x   39 daniel  staff   1.2K Dec 30 15:48 TH
drwxr-xr-x    4 daniel  staff   128B Nov  1 16:26 THNN
drwxr-xr-x   14 daniel  staff   448B Dec 30 15:48 c10
drwxr-xr-x    5 daniel  staff   160B Dec 30 15:48 caffe2
drwxr-xr-x   21 daniel  staff   672B Dec 30 15:48 pybind11
drwxr-xr-x    6 daniel  staff   192B Dec 30 15:48 torch

So there are TH and THNN folders, but no THC. Can this something specific to my operating system?

ctongfei commented 5 years ago

Yes this is a Mac issue -- the standard installation of PyTorch does not support CUDA on Mac. You can either install PyTorch from source on Mac (<Mojave; Mojave stopped supporting CUDA), or use Linux. This script runs fine on Linux (at least for me).

You can tweak the build.sh script to remove CUDA dependencies, so that it would build a pure CPU Torch binding for the JVM.

danyaljj commented 5 years ago

An update on this with this, with the new make file;

This is the try: 

Daniels-MBP-3:torch daniel$ make
PyTorch 1.0.0 found at /Users/daniel/miniconda3/lib/python3.7/site-packages/torch
OS is Darwin
JAVA_HOME is 
CUDA_HOME is 
mkdir -p ./include-original;
for m in TH THC ATen; do \
        cp -R /Users/daniel/miniconda3/lib/python3.7/site-packages/torch/lib/include/$m ./include-original/$m; \
    done;
cp: /Users/daniel/miniconda3/lib/python3.7/site-packages/torch/lib/include/THC: No such file or directory
/Users/daniel/miniconda3/lib/python3.7/site-packages/torch/lib
cp /Users/daniel/miniconda3/lib/python3.7/site-packages/torch/lib/THNN.h ./include-original;
cp /Users/daniel/miniconda3/lib/python3.7/site-packages/torch/lib/THCUNN.h ./include-original;
mkdir -p ./include-swig; \
    cp -R ./include-original/* ./include-swig; \
    for f in ./include-original/THNN.h ./include-original/THCUNN.h ./include-original/ATen/Type.h ./include-original/ATen/Formatting.h ./include-original/ATen/CPUHalfTensor.h ./include-original/ATen/Error.h ./include-original/ATen/Half.h ./include-original/ATen/CPUShortTensor.h ./include-original/ATen/Utils.h ./include-original/ATen/TensorOptions.h ./include-original/ATen/CPUDoubleTensor.h ./include-original/ATen/TensorUtils.h ./include-original/ATen/InitialTensorOptions.h ./include-original/ATen/CPUShortStorage.h ./include-original/ATen/Retainable.h ./include-original/ATen/DLConvertor.h ./include-original/ATen/AlignOf.h ./include-original/ATen/Device.h ./include-original/ATen/SparseCPUIntType.h ./include-original/ATen/SparseCUDALongType.h ./include-original/ATen/core/Type.h ./include-original/ATen/core/Formatting.h ./include-original/ATen/core/Half.h ./include-original/ATen/core/TensorOptions.h ./include-original/ATen/core/typeid.h ./include-original/ATen/core/Range.h ./include-original/ATen/core/interned_strings_class.h ./include-original/ATen/core/DefaultDtype.h ./include-original/ATen/core/LegacyDeviceTypeInit.h ./include-original/ATen/core/Backtrace.h ./include-original/ATen/core/context_base.h ./include-original/ATen/core/blob.h ./include-original/ATen/core/function_schema.h ./include-original/ATen/core/WrapDimMinimal.h ./include-original/ATen/core/aten_interned_strings.h ./include-original/ATen/core/LegacyTypeDispatch.h ./include-original/ATen/core/UndefinedTensorImpl.h ./include-original/ATen/core/Scalar.h ./include-original/ATen/core/SparseTensorRef.h ./include-original/ATen/core/functional.h ./include-original/ATen/core/thread_pool.h ./include-original/ATen/core/interned_strings.h ./include-original/ATen/core/TensorImpl.h ./include-original/ATen/core/TensorMethods.h ./include-original/ATen/core/Macros.h ./include-original/ATen/core/VariableHooksInterface.h ./include-original/ATen/core/ScalarType.h ./include-original/ATen/core/ATenGeneral.h ./include-original/ATen/core/Deprecated.h ./include-original/ATen/core/alias_info.h ./include-original/ATen/core/jit_type.h ./include-original/ATen/core/ivalue.h ./include-original/ATen/core/DimVector.h ./include-original/ATen/core/Reduction.h ./include-original/ATen/core/Tensor.h ./include-original/ATen/core/Generator.h ./include-original/ATen/core/TensorAccessor.h ./include-original/ATen/CPUIntType.h ./include-original/ATen/CPUFloatType.h ./include-original/ATen/CPUCharStorage.h ./include-original/ATen/dlpack.h ./include-original/ATen/Half-inl.h ./include-original/ATen/Config.h ./include-original/ATen/CPUCharTensor.h ./include-original/ATen/SparseCUDACharType.h ./include-original/ATen/SparseTensorUtils.h ./include-original/ATen/Backtrace.h ./include-original/ATen/CPUShortType.h ./include-original/ATen/cpu/FlushDenormal.h ./include-original/ATen/cpu/vml.h ./include-original/ATen/SparseCPULongType.h ./include-original/ATen/CUDALongStorage.h ./include-original/ATen/CUDAStream.h ./include-original/ATen/CPUHalfStorage.h ./include-original/ATen/Backend.h ./include-original/ATen/CUDAFloatTensor.h ./include-original/ATen/CPUGenerator.h ./include-original/ATen/CUDALongType.h ./include-original/ATen/SparseCPUFloatType.h ./include-original/ATen/cuda/CUDADevice.h ./include-original/ATen/cuda/CUDAUtils.h ./include-original/ATen/cuda/CUDAConfig.h ./include-original/ATen/cuda/PinnedMemoryAllocator.h ./include-original/ATen/cuda/CUDAMultiStreamGuard.h ./include-original/ATen/cuda/CUDAEvent.h ./include-original/ATen/cuda/CUDAStream.h ./include-original/ATen/cuda/CUDATypeDefault.h ./include-original/ATen/cuda/CUDAContext.h ./include-original/ATen/cuda/CUDAGuard.h ./include-original/ATen/cuda/Exceptions.h ./include-original/ATen/cuda/_curand_mtgp32_host.h ./include-original/ATen/cuda/detail/CUDAHooks.h ./include-original/ATen/cuda/detail/KernelUtils.h ./include-original/ATen/cuda/detail/CUDAGuardImpl.h ./include-original/ATen/cuda/Array.h ./include-original/ATen/cuda/ATenCUDAGeneral.h ./include-original/ATen/SparseCPUCharType.h ./include-original/ATen/CUDAIntTensor.h ./include-original/ATen/CPUByteStorage.h ./include-original/ATen/TypeExtendedInterface.h ./include-original/ATen/WrapDimUtils.h ./include-original/ATen/CPUIntStorage.h ./include-original/ATen/ScalarTypeUtils.h ./include-original/ATen/LegacyTHDispatch.h ./include-original/ATen/CPUHalfType.h ./include-original/ATen/CUDACharType.h ./include-original/ATen/Context.h ./include-original/ATen/SparseCPUShortType.h ./include-original/ATen/CUDADoubleStorage.h ./include-original/ATen/CUDADoubleType.h ./include-original/ATen/div_rtn.h ./include-original/ATen/ExpandUtils.h ./include-original/ATen/CPULongType.h ./include-original/ATen/TypeDefault.h ./include-original/ATen/CUDALongTensor.h ./include-original/ATen/CPUFixedAllocator.h ./include-original/ATen/CUDAHalfType.h ./include-original/ATen/CPUCharType.h ./include-original/ATen/Registry.h ./include-original/ATen/optional.h ./include-original/ATen/Scalar.h ./include-original/ATen/SparseTensorRef.h ./include-original/ATen/CPUByteTensor.h ./include-original/ATen/CheckGenerator.h ./include-original/ATen/ArrayRef.h ./include-original/ATen/CUDAFloatStorage.h ./include-original/ATen/MatrixRef.h ./include-original/ATen/ATen.h ./include-original/ATen/CUDADoubleTensor.h ./include-original/ATen/CUDAFloatType.h ./include-original/ATen/Layout.h ./include-original/ATen/SparseTensorImpl.h ./include-original/ATen/detail/CPUGuardImpl.h ./include-original/ATen/detail/ScalarTypeConversions.h ./include-original/ATen/detail/CUDAHooksInterface.h ./include-original/ATen/detail/ComplexHooksInterface.h ./include-original/ATen/detail/FunctionTraits.h ./include-original/ATen/detail/HIPHooksInterface.h ./include-original/ATen/detail/VariableHooksInterface.h ./include-original/ATen/detail/UniqueVoidPtr.h ./include-original/ATen/SparseCPUDoubleType.h ./include-original/ATen/UndefinedTensor.h ./include-original/ATen/RegisterCUDA.h ./include-original/ATen/SparseCUDAByteType.h ./include-original/ATen/CPUGeneral.h ./include-original/ATen/WrapDimUtilsMulti.h ./include-original/ATen/Allocator.h ./include-original/ATen/CUDACharTensor.h ./include-original/ATen/OptionsGuard.h ./include-original/ATen/TensorImpl.h ./include-original/ATen/TensorMethods.h ./include-original/ATen/CUDAShortType.h ./include-original/ATen/CUDAHalfTensor.h ./include-original/ATen/TensorOperators.h ./include-original/ATen/ScalarType.h ./include-original/ATen/Storage.h ./include-original/ATen/DeviceGuard.h ./include-original/ATen/SparseCUDADoubleType.h ./include-original/ATen/CUDAByteStorage.h ./include-original/ATen/ATenGeneral.h ./include-original/ATen/CPUFloatTensor.h ./include-original/ATen/Deprecated.h ./include-original/ATen/Dispatch.h ./include-original/ATen/CPULongStorage.h ./include-original/ATen/Functions.h ./include-original/ATen/TensorBase.h ./include-original/ATen/CUDAShortStorage.h ./include-original/ATen/CUDAHalfStorage.h ./include-original/ATen/CPUTypeDefault.h ./include-original/ATen/RegisterCPU.h ./include-original/ATen/SparseCUDAIntType.h ./include-original/ATen/CPUDoubleType.h ./include-original/ATen/CPUApplyUtils.h ./include-original/ATen/ScalarOps.h ./include-original/ATen/NativeFunctions.h ./include-original/ATen/CPULongTensor.h ./include-original/ATen/TensorGeometry.h ./include-original/ATen/SparseCUDAShortType.h ./include-original/ATen/CPUByteType.h ./include-original/ATen/CUDACharStorage.h ./include-original/ATen/THLongStorageView.h ./include-original/ATen/SparseCUDAFloatType.h ./include-original/ATen/CUDAByteTensor.h ./include-original/ATen/LegacyTHDispatcher.h ./include-original/ATen/CUDAIntType.h ./include-original/ATen/CPUFloatStorage.h ./include-original/ATen/Parallel.h ./include-original/ATen/DimVector.h ./include-original/ATen/InferSize.h ./include-original/ATen/CUDAGenerator.h ./include-original/ATen/SmallVector.h ./include-original/ATen/Tensor.h ./include-original/ATen/UndefinedType.h ./include-original/ATen/SparseCPUByteType.h ./include-original/ATen/CUDAIntStorage.h ./include-original/ATen/Generator.h ./include-original/ATen/CPUIntTensor.h ./include-original/ATen/CUDAShortTensor.h ./include-original/ATen/AccumulateType.h ./include-original/ATen/CPUDoubleStorage.h ./include-original/ATen/CUDAByteType.h ./include-original/ATen/TensorAccessor.h ./include-original/ATen/cudnn/Handles.h ./include-original/ATen/cudnn/Utils.h ./include-original/ATen/cudnn/Types.h ./include-original/ATen/cudnn/Descriptors.h ./include-original/ATen/cudnn/Exceptions.h ./include-original/ATen/cudnn/Handle.h ./include-original/ATen/cudnn/cudnn-wrapper.h ./include-original/TH/THTensorDimApply.h ./include-original/TH/THBlas.h ./include-original/TH/THFilePrivate.h ./include-original/TH/THGenerateFloatType.h ./include-original/TH/THFile.h ./include-original/TH/THGenerateDoubleType.h ./include-original/TH/THGenerateShortType.h ./include-original/TH/THGenerateIntTypes.h ./include-original/TH/THGenerateLongType.h ./include-original/TH/THAllocator.h ./include-original/TH/THGenerateCharType.h ./include-original/TH/THSize.h ./include-original/TH/THMemoryFile.h ./include-original/TH/THStorage.h ./include-original/TH/THHalf.h ./include-original/TH/THGenerateHalfType.h ./include-original/TH/THGenerateIntType.h ./include-original/TH/THVector.h ./include-original/TH/THGeneral.h ./include-original/TH/THLapack.h ./include-original/TH/THRandom.h ./include-original/TH/THLogAdd.h ./include-original/TH/THGenerateFloatTypes.h ./include-original/TH/generic/THBlas.h ./include-original/TH/generic/THTensorMath.h ./include-original/TH/generic/THTensorConv.h ./include-original/TH/generic/THStorage.h ./include-original/TH/generic/THTensorCopy.h ./include-original/TH/generic/THTensorLapack.h ./include-original/TH/generic/THTensorRandom.h ./include-original/TH/generic/THVector.h ./include-original/TH/generic/THStorageCopy.h ./include-original/TH/generic/THLapack.h ./include-original/TH/generic/THTensor.h ./include-original/TH/THTensor.h ./include-original/TH/THMath.h ./include-original/TH/THDiskFile.h ./include-original/TH/TH.h ./include-original/TH/THTensorApply.h ./include-original/TH/THGenerateAllTypes.h ./include-original/TH/THGenerateByteType.h ./include-original/TH/THStorageFunctions.h; do \
        x=${f#./include-original/}; \
        cat $f \
        | sed -E "s|<TH(.*)>|\"TH\1\"|g" \
        | grep -v "#include <.*>" \
        | grep -v "#include \"cu.*\.h\"" \
        | grep -v "TH_NO_RETURN" \
        | grep -v "__signed" \
        | grep -v "Reduction.h" \
        | sed -e "s|__thalign__([0-9])||g" \
        > ./include-swig/$x; \
    done
g++ -P -E \
      -I include-swig \
      -I include-swig/TH \
      -I include-swig/THNN \
      -I include-swig/THC \
      -I include-swig/THCUNN \
      torch.h \
      | sed -e 's/__attribute__((__visibility__("default")))//g' \
      | sed -e 's/CAFFE2_API//g' \
      | grep -v "AT_CUDA_API" \
      | grep -v "^at::DataPtr" \
      | python remove_classes.py \
      > torch-preprocessed.h
clang: warning: treating 'c-header' input as 'c++-header' when in C++ mode, this behavior is deprecated [-Wdeprecated]
torch.h:2:10: fatal error: 'THC/THC.h' file not found
#include "THC/THC.h"
         ^~~~~~~~~~~
1 error generated.
mkdir -p jni/src/main/java/nexus/torch/jni;
swig -c++ -v -DSWIGWORDSIZE64 -java \
      -package nexus.torch.jni \
      -outdir jni/src/main/java/nexus/torch/jni torch.i
Language subdirectory: java
Search paths:
   ./
   ./swig_lib/java/
   /usr/local/share/swig/3.0.12/java/
   ./swig_lib/
   /usr/local/share/swig/3.0.12/
Preprocessing...
Starting language-specific parse...
Processing types...
C++ analysis...
Processing nested classes...
Generating wrappers...
cat torch_wrap.cxx \
      | python fix_cuda_stream_dereferencing.py \
      > torch_wrap_fixed.cxx
g++  -c torch_wrap_fixed.cxx \
    -I /include \
    -I /include/darwin \
    -I /include \
    -I /Users/daniel/miniconda3/lib/python3.7/site-packages/torch/lib \
    -I /Users/daniel/miniconda3/lib/python3.7/site-packages/torch/lib/include \
    -I /Users/daniel/miniconda3/lib/python3.7/site-packages/torch/lib/include/TH \
    -I /Users/daniel/miniconda3/lib/python3.7/site-packages/torch/lib/include/THC
torch_wrap_fixed.cxx:178:10: fatal error: 'jni.h' file not found
#include <jni.h>
         ^~~~~~~
1 error generated.

Turns out that both JAVA_HOME and CUDA_HOME are missing. After setting these variables, the error changes to the following: 

Daniels-MBP-3:torch daniel$ make 
PyTorch 1.0.0 found at /Users/daniel/miniconda3/lib/python3.7/site-packages/torch
OS is Darwin
JAVA_HOME is /Library/Java/JavaVirtualMachines/jdk1.8.0_72.jdk/Contents/Home/
CUDA_HOME is /usr/local/cuda
g++  -c torch_wrap_fixed.cxx \
    -I /Library/Java/JavaVirtualMachines/jdk1.8.0_72.jdk/Contents/Home//include \
    -I /Library/Java/JavaVirtualMachines/jdk1.8.0_72.jdk/Contents/Home//include/darwin \
    -I /usr/local/cuda/include \
    -I /Users/daniel/miniconda3/lib/python3.7/site-packages/torch/lib \
    -I /Users/daniel/miniconda3/lib/python3.7/site-packages/torch/lib/include \
    -I /Users/daniel/miniconda3/lib/python3.7/site-packages/torch/lib/include/TH \
    -I /Users/daniel/miniconda3/lib/python3.7/site-packages/torch/lib/include/THC
In file included from torch_wrap_fixed.cxx:237:
In file included from ./include/TH/TH.h:15:
In file included from ./include/TH/THStorageFunctions.h:4:
In file included from ./include/TH/THAllocator.h:6:
In file included from /Users/daniel/miniconda3/lib/python3.7/site-packages/torch/lib/include/ATen/Allocator.h:2:
In file included from /Users/daniel/miniconda3/lib/python3.7/site-packages/torch/lib/include/c10/core/Allocator.h:6:
In file included from /Users/daniel/miniconda3/lib/python3.7/site-packages/torch/lib/include/c10/Device.h:3:
/Users/daniel/miniconda3/lib/python3.7/site-packages/torch/lib/include/c10/DeviceType.h:15:6: warning: scoped enumerations are a C++11 extension [-Wc++11-extensions]
enum class DeviceType : int16_t {
     ^
/Users/daniel/miniconda3/lib/python3.7/site-packages/torch/lib/include/c10/DeviceType.h:15:12: error: ISO C++ forbids forward references to 'enum' types
enum class DeviceType : int16_t {
           ^
/Users/daniel/miniconda3/lib/python3.7/site-packages/torch/lib/include/c10/DeviceType.h:15:23: error: expected unqualified-id
enum class DeviceType : int16_t {
                      ^
/Users/daniel/miniconda3/lib/python3.7/site-packages/torch/lib/include/c10/DeviceType.h:33:1: error: unknown type name 'constexpr'
constexpr int COMPILE_TIME_MAX_DEVICE_TYPES =
^
/Users/daniel/miniconda3/lib/python3.7/site-packages/torch/lib/include/c10/DeviceType.h:33:11: error: expected unqualified-id
constexpr int COMPILE_TIME_MAX_DEVICE_TYPES =
          ^
In file included from torch_wrap_fixed.cxx:237:
In file included from ./include/TH/TH.h:15:
In file included from ./include/TH/THStorageFunctions.h:4:
In file included from ./include/TH/THAllocator.h:6:
In file included from /Users/daniel/miniconda3/lib/python3.7/site-packages/torch/lib/include/ATen/Allocator.h:2:
In file included from /Users/daniel/miniconda3/lib/python3.7/site-packages/torch/lib/include/c10/core/Allocator.h:6:
In file included from /Users/daniel/miniconda3/lib/python3.7/site-packages/torch/lib/include/c10/Device.h:5:
In file included from /Users/daniel/miniconda3/lib/python3.7/site-packages/torch/lib/include/c10/util/Exception.h:5:
/Users/daniel/miniconda3/lib/python3.7/site-packages/torch/lib/include/c10/util/StringUtil.h:30:31: warning: variadic templates are a C++11 extension [-Wc++11-extensions]
template <typename T, typename... Args>
                              ^
/Users/daniel/miniconda3/lib/python3.7/site-packages/torch/lib/include/c10/util/StringUtil.h:38:19: warning: variadic templates are a C++11 extension [-Wc++11-extensions]
template <typename... Args>
                  ^
/Users/daniel/miniconda3/lib/python3.7/site-packages/torch/lib/include/c10/util/StringUtil.h:58:8: warning: 'auto' type specifier is a C++11 extension [-Wc++11-extensions]
  for (auto i = v.begin(); i != v.end(); ++i, --cnt) {
       ^
In file included from torch_wrap_fixed.cxx:237:
In file included from ./include/TH/TH.h:15:
In file included from ./include/TH/THStorageFunctions.h:4:
In file included from ./include/TH/THAllocator.h:6:
In file included from /Users/daniel/miniconda3/lib/python3.7/site-packages/torch/lib/include/ATen/Allocator.h:2:
In file included from /Users/daniel/miniconda3/lib/python3.7/site-packages/torch/lib/include/c10/core/Allocator.h:6:
In file included from /Users/daniel/miniconda3/lib/python3.7/site-packages/torch/lib/include/c10/Device.h:5:
/Users/daniel/miniconda3/lib/python3.7/site-packages/torch/lib/include/c10/util/Exception.h:70:15: error: exception specification of overriding function is more lax than base version
  const char* what() const noexcept override {
              ^
/Library/Developer/CommandLineTools/usr/include/c++/v1/exception:102:25: note: overridden virtual function is here
    virtual const char* what() const _NOEXCEPT;
                        ^
In file included from torch_wrap_fixed.cxx:237:
In file included from ./include/TH/TH.h:15:
In file included from ./include/TH/THStorageFunctions.h:4:
In file included from ./include/TH/THAllocator.h:6:
In file included from /Users/daniel/miniconda3/lib/python3.7/site-packages/torch/lib/include/ATen/Allocator.h:2:
In file included from /Users/daniel/miniconda3/lib/python3.7/site-packages/torch/lib/include/c10/core/Allocator.h:6:
In file included from /Users/daniel/miniconda3/lib/python3.7/site-packages/torch/lib/include/c10/Device.h:5:
/Users/daniel/miniconda3/lib/python3.7/site-packages/torch/lib/include/c10/util/Exception.h:70:27: error: expected ';' at end of declaration list
  const char* what() const noexcept override {
                          ^
                          ;
/Users/daniel/miniconda3/lib/python3.7/site-packages/torch/lib/include/c10/util/Exception.h:27:15: error: exception specification of overriding function is more lax than base version
class C10_API Error : public std::exception {
              ^
/Users/daniel/miniconda3/lib/python3.7/site-packages/torch/lib/include/c10/util/Exception.h:27:15: note: while declaring the implicit destructor for 'Error'
/Library/Developer/CommandLineTools/usr/include/c++/v1/exception:101:13: note: overridden virtual function is here
    virtual ~exception() _NOEXCEPT;
            ^
In file included from torch_wrap_fixed.cxx:237:
In file included from ./include/TH/TH.h:15:
In file included from ./include/TH/THStorageFunctions.h:4:
In file included from ./include/TH/THAllocator.h:6:
In file included from /Users/daniel/miniconda3/lib/python3.7/site-packages/torch/lib/include/ATen/Allocator.h:2:
In file included from /Users/daniel/miniconda3/lib/python3.7/site-packages/torch/lib/include/c10/core/Allocator.h:6:
In file included from /Users/daniel/miniconda3/lib/python3.7/site-packages/torch/lib/include/c10/Device.h:5:
/Users/daniel/miniconda3/lib/python3.7/site-packages/torch/lib/include/c10/util/Exception.h:86:7: warning: alias declarations are a C++11 extension [-Wc++11-extensions]
      void (*)(const SourceLocation& source_location, const char* msg);
      ^
In file included from torch_wrap_fixed.cxx:237:
In file included from ./include/TH/TH.h:15:
In file included from ./include/TH/THStorageFunctions.h:4:
In file included from ./include/TH/THAllocator.h:6:
In file included from /Users/daniel/miniconda3/lib/python3.7/site-packages/torch/lib/include/ATen/Allocator.h:2:
In file included from /Users/daniel/miniconda3/lib/python3.7/site-packages/torch/lib/include/c10/core/Allocator.h:6:
/Users/daniel/miniconda3/lib/python3.7/site-packages/torch/lib/include/c10/Device.h:18:21: warning: alias declarations are a C++11 extension [-Wc++11-extensions]
using DeviceIndex = int16_t;
                    ^
/Users/daniel/miniconda3/lib/python3.7/site-packages/torch/lib/include/c10/Device.h:30:23: warning: 'final' keyword is a C++11 extension [-Wc++11-extensions]
struct C10_API Device final {
                      ^
/Users/daniel/miniconda3/lib/python3.7/site-packages/torch/lib/include/c10/Device.h:31:16: warning: alias declarations are a C++11 extension [-Wc++11-extensions]
  using Type = DeviceType;
               ^
/Users/daniel/miniconda3/lib/python3.7/site-packages/torch/lib/include/c10/Device.h:56:45: error: expected ';' at end of declaration list
  bool operator==(const Device& other) const noexcept {
                                            ^
                                            ;
/Users/daniel/miniconda3/lib/python3.7/site-packages/torch/lib/include/c10/Device.h:98:22: warning: in-class initialization of non-static data member is a C++11 extension
      [-Wc++11-extensions]
  DeviceIndex index_ = -1;
                     ^
/Users/daniel/miniconda3/lib/python3.7/site-packages/torch/lib/include/c10/Device.h:36:9: error: member initializer 'type_' does not name a non-static data member or base class
      : type_(type), index_(index) {
        ^~~~~~~~~~~
/Users/daniel/miniconda3/lib/python3.7/site-packages/torch/lib/include/c10/Device.h:37:5: error: expected expression
    AT_CHECK(
    ^
/Users/daniel/miniconda3/lib/python3.7/site-packages/torch/lib/include/c10/util/Exception.h:150:5: note: expanded from macro 'AT_CHECK'
    AT_ERROR(::c10::str(__VA_ARGS__)); \
    ^
/Users/daniel/miniconda3/lib/python3.7/site-packages/torch/lib/include/c10/util/Exception.h:121:22: note: expanded from macro 'AT_ERROR'
  throw ::c10::Error({__func__, __FILE__, __LINE__}, ::c10::str(__VA_ARGS__))
                     ^
In file included from torch_wrap_fixed.cxx:237:
In file included from ./include/TH/TH.h:15:
In file included from ./include/TH/THStorageFunctions.h:4:
In file included from ./include/TH/THAllocator.h:6:
In file included from /Users/daniel/miniconda3/lib/python3.7/site-packages/torch/lib/include/ATen/Allocator.h:2:
In file included from /Users/daniel/miniconda3/lib/python3.7/site-packages/torch/lib/include/c10/core/Allocator.h:6:
/Users/daniel/miniconda3/lib/python3.7/site-packages/torch/lib/include/c10/Device.h:42:10: error: use of undeclared identifier 'is_cpu'
        !is_cpu() || index <= 0,
         ^
/Users/daniel/miniconda3/lib/python3.7/site-packages/torch/lib/include/c10/Device.h:41:5: error: expected expression
    AT_CHECK(
    ^
/Users/daniel/miniconda3/lib/python3.7/site-packages/torch/lib/include/c10/util/Exception.h:150:5: note: expanded from macro 'AT_CHECK'
    AT_ERROR(::c10::str(__VA_ARGS__)); \
    ^
/Users/daniel/miniconda3/lib/python3.7/site-packages/torch/lib/include/c10/util/Exception.h:121:22: note: expanded from macro 'AT_ERROR'
  throw ::c10::Error({__func__, __FILE__, __LINE__}, ::c10::str(__VA_ARGS__))
                     ^
In file included from torch_wrap_fixed.cxx:237:
In file included from ./include/TH/TH.h:15:
In file included from ./include/TH/THStorageFunctions.h:4:
In file included from ./include/TH/THAllocator.h:6:
In file included from /Users/daniel/miniconda3/lib/python3.7/site-packages/torch/lib/include/ATen/Allocator.h:2:
In file included from /Users/daniel/miniconda3/lib/python3.7/site-packages/torch/lib/include/c10/core/Allocator.h:6:
/Users/daniel/miniconda3/lib/python3.7/site-packages/torch/lib/include/c10/Device.h:110:41: error: expected ';' at end of declaration list
  size_t operator()(c10::Device d) const noexcept {
                                        ^
                                        ;
In file included from torch_wrap_fixed.cxx:237:
In file included from ./include/TH/TH.h:15:
In file included from ./include/TH/THStorageFunctions.h:4:
In file included from ./include/TH/THAllocator.h:6:
In file included from /Users/daniel/miniconda3/lib/python3.7/site-packages/torch/lib/include/ATen/Allocator.h:2:
In file included from /Users/daniel/miniconda3/lib/python3.7/site-packages/torch/lib/include/c10/core/Allocator.h:7:
/Users/daniel/miniconda3/lib/python3.7/site-packages/torch/lib/include/c10/util/UniqueVoidPtr.h:8:22: warning: alias declarations are a C++11 extension [-Wc++11-extensions]
using DeleterFnPtr = void (*)(void*);
                     ^
/Users/daniel/miniconda3/lib/python3.7/site-packages/torch/lib/include/c10/util/UniqueVoidPtr.h:66:38: warning: rvalue references are a C++11 extension [-Wc++11-extensions]
  std::unique_ptr<void, DeleterFnPtr>&& move_context() {
                                     ^
/Users/daniel/miniconda3/lib/python3.7/site-packages/torch/lib/include/c10/util/UniqueVoidPtr.h:67:12: warning: returning reference to local temporary object [-Wreturn-stack-address]
    return std::move(ctx_);
           ^~~~~~~~~~~~~~~
/Users/daniel/miniconda3/lib/python3.7/site-packages/torch/lib/include/c10/util/UniqueVoidPtr.h:102:65: error: expected function body after function declarator
inline bool operator==(const UniqueVoidPtr& sp, std::nullptr_t) noexcept {
                                                                ^
In file included from torch_wrap_fixed.cxx:237:
In file included from ./include/TH/TH.h:15:
In file included from ./include/TH/THStorageFunctions.h:4:
In file included from ./include/TH/THAllocator.h:6:
In file included from /Users/daniel/miniconda3/lib/python3.7/site-packages/torch/lib/include/ATen/Allocator.h:2:
/Users/daniel/miniconda3/lib/python3.7/site-packages/torch/lib/include/c10/core/Allocator.h:46:38: warning: rvalue references are a C++11 extension [-Wc++11-extensions]
  std::unique_ptr<void, DeleterFnPtr>&& move_context() {
                                     ^
/Users/daniel/miniconda3/lib/python3.7/site-packages/torch/lib/include/c10/core/Allocator.h:27:31: warning: use of enumeration in a nested name specifier is a C++11 extension
      [-Wc++11-extensions]
  DataPtr() : ptr_(), device_(DeviceType::CPU) {}
                              ^
/Users/daniel/miniconda3/lib/python3.7/site-packages/torch/lib/include/c10/core/Allocator.h:27:31: error: incomplete type 'c10::DeviceType' named in nested name specifier
  DataPtr() : ptr_(), device_(DeviceType::CPU) {}
                              ^~~~~~~~~~~~
/Users/daniel/miniconda3/lib/python3.7/site-packages/torch/lib/include/c10/core/Allocator.h:67:59: error: expected function body after function declarator
inline bool operator==(const DataPtr& dp, std::nullptr_t) noexcept {
                                                          ^
/Users/daniel/miniconda3/lib/python3.7/site-packages/torch/lib/include/c10/core/Allocator.h:126:48: error: expected '(' for function-style cast or type construction
  std::unique_ptr<void, std::function<void(void*)>> ptr_;
                                           ~~~~^
/Users/daniel/miniconda3/lib/python3.7/site-packages/torch/lib/include/c10/core/Allocator.h:126:49: error: expected expression
  std::unique_ptr<void, std::function<void(void*)>> ptr_;
                                                ^
/Users/daniel/miniconda3/lib/python3.7/site-packages/torch/lib/include/c10/core/Allocator.h:126:53: error: use of undeclared identifier 'ptr_'; did you mean 'str'?
  std::unique_ptr<void, std::function<void(void*)>> ptr_;
                                                    ^~~~
                                                    str
/Users/daniel/miniconda3/lib/python3.7/site-packages/torch/lib/include/c10/util/StringUtil.h:39:20: note: 'str' declared here
inline std::string str(const Args&... args) {
                   ^
fatal error: too many errors emitted, stopping now [-ferror-limit=]
14 warnings and 20 errors generated.
make: *** [torch_wrap_fixed.o] Error 1
ctongfei commented 5 years ago

Seems that you need -std=c++11 enabled in your C++ compiler on Mac? Probably putting that in $(CXXFLAGS_Darwin) in the Makefile?

danyaljj commented 5 years ago

Indeed that was a problem. I added that and the following error appeared: 

Daniels-MBP-3:torch daniel$ make 
PyTorch 1.0.0 found at /Users/daniel/miniconda3/lib/python3.7/site-packages/torch
OS is Darwin
JAVA_HOME is /Library/Java/JavaVirtualMachines/jdk1.8.0_72.jdk/Contents/Home/
CUDA_HOME is /usr/local/cuda
g++ -std=c++11 -c torch_wrap_fixed.cxx \
    -I /Library/Java/JavaVirtualMachines/jdk1.8.0_72.jdk/Contents/Home//include \
    -I /Library/Java/JavaVirtualMachines/jdk1.8.0_72.jdk/Contents/Home//include/darwin \
    -I /usr/local/cuda/include \
    -I /Users/daniel/miniconda3/lib/python3.7/site-packages/torch/lib \
    -I /Users/daniel/miniconda3/lib/python3.7/site-packages/torch/lib/include \
    -I /Users/daniel/miniconda3/lib/python3.7/site-packages/torch/lib/include/TH \
    -I /Users/daniel/miniconda3/lib/python3.7/site-packages/torch/lib/include/THC
torch_wrap_fixed.cxx:238:14: fatal error: 'include/THC/THC.h' file not found
    #include "include/THC/THC.h"
             ^~~~~~~~~~~~~~~~~~~
1 error generated.
make: *** [torch_wrap_fixed.o] Error 1

which is likely because there is no THC folder in my PyTorch dist: 

Daniels-MBP-3:torch daniel$ ls  /Users/daniel/miniconda3/lib/python3.7/site-packages/torch/lib/include/
ATen        TH      THNN        c10     caffe2      pybind11    torch
ctongfei commented 5 years ago

@danyaljj Remove all CUDA references in your Makefile. PyTorch on Mac, unless built from source, does not have CUDA support.

danyaljj commented 5 years ago

Ah that feels like a non-trivial change. My understanding is that to do this we should drop:

Does that sound reasonable to you?

ctongfei commented 5 years ago

That's right -- I don't want to use autoconf (.configure then make) -- that's too much hassle. If you'd like you can modify the Makefile to check if $(CUDA_HOME) is null before proceeding with each task.