apache / mxnet

Lightweight, Portable, Flexible Distributed/Mobile Deep Learning with Dynamic, Mutation-aware Dataflow Dep Scheduler; for Python, R, Julia, Scala, Go, Javascript and more
https://mxnet.apache.org
Apache License 2.0
20.78k stars 6.79k forks source link

[BUG] config for windows? #19048

Open ekdnam opened 4 years ago

ekdnam commented 4 years ago

Description

(A clear and concise description of what the bug is.) While building MXNet from source (please check: https://github.com/apache/incubator-mxnet/issues/19002), no config is given for windows OS here https://github.com/apache/incubator-mxnet/tree/master/config. What is to be done in such a case? Are some updates required in the documentation? Or should one use cmake cmakelists.txt?

Error Message

(Paste the complete error message. Please also include stack trace by setting environment variable DMLC_LOG_STACK_TRACE_DEPTH=10 before running your script.)

To Reproduce

(If you developed your own code, please provide a short script that reproduces the error. For existing examples, please provide link.)

Steps to reproduce

(Paste the commands you ran that produced the error.)

1. 2.

What have you tried to solve it?

1. 2.

Environment

We recommend using our script for collecting the diagnositc information. Run the following command and paste the outputs below:

curl --retry 10 -s https://raw.githubusercontent.com/apache/incubator-mxnet/master/tools/diagnose.py | python

# paste outputs here
szha commented 4 years ago

thanks for bringing this up. cc @yajiedesign @leezu for suggestions

leezu commented 4 years ago

You can start with the linux config and see if it builds without errors. If there are errors, you can post here. Once it works, you can create a PR with the windows config file.

Essentially the config is just a convenience feature to change the default options. If you don't specify the config, the defaults from cmakelists.txt will be used (ie. when running cmake ..)

yajiedesign commented 4 years ago

cmake option is cached.if you change CMakeLists.txt.must clear cmake cache.(delete CMakeCache.txt and CMakeFiles).

ekdnam commented 4 years ago

Will surely do the needful, and update you all if there are some errors. Thanks a lot.

ekdnam commented 4 years ago

After the step cmake .., I am getting the following error:

CMake Error at C:/Program Files/CMake/share/cmake-3.18/Modules/FindPackageHandleStandardArgs.cmake:165 (message):
  Could NOT find Accelerate (missing: Accelerate_CBLAS_INCLUDE_DIR)
Call Stack (most recent call first):
  C:/Program Files/CMake/share/cmake-3.18/Modules/FindPackageHandleStandardArgs.cmake:458 (_FPHSA_FAILURE_MESSAGE)
  cmake/Modules/FindAccelerate.cmake:39 (find_package_handle_standard_args)
  cmake/ChooseBlas.cmake:56 (find_package)
  CMakeLists.txt:285 (include)

What is to be done?

leezu commented 4 years ago

You can set the BLAS to be used in the config file.

The install guide recommends using OpenBLAS: "You may download it from from the OpenBLAS release page and compile from source. Set the OpenBLAS_HOME environment variable to point to the OpenBLAS directory that contains the include and lib directories for example by typing set OpenBLAS_HOME=C:\utils\OpenBLAS."

So two steps: 1) install openblas 2) set openblas as blas in the config file

ekdnam commented 4 years ago

I have followed the steps given in the install guide, and have set the environment variable. Screenshots below

openblas-mxnet

openblas-dir

leezu commented 4 years ago

Thank you @ekdnam. Does the compilation work for you now?

ekdnam commented 4 years ago

After the step cmake .., I am getting the following error:

I had set OpenBLAS as an environment variable when this error was posted. The compilation is still not working :(

leezu commented 4 years ago

So have you set the set(USE_BLAS "open" CACHE STRING "BLAS Vendor") option in the config file?

ekdnam commented 4 years ago

I have to do this, right?

set(USE_BLAS ON "open" CACHE STRING "BLAS Vendor")

And apologies if I am troubling y'all in your work. I am new to MXNet, and am very, very excited to work with it. :D

ekdnam commented 4 years ago

In the file cmake\Modules\FindOpenBLAS.cmake, there are paths OpenBLAS_HOME, and OpenBLAS_DIR, here

https://github.com/apache/incubator-mxnet/blob/f2e90a22c7190e86441a415b1dbdb47ad3603f30/cmake/Modules/FindOpenBLAS.cmake#L18-L19

What should be the path OpenBLAS_DIR? Should it be set as an environment variable?

leezu commented 4 years ago

Setting set(USE_BLAS ON "open" CACHE STRING "BLAS Vendor") in the config file or adding the -DUSE_BLAS=open option to the command line when calling cmake should be sufficient.

ekdnam commented 4 years ago

I cloned the repo once again, followed the entire procedure of building from start to end, and now atleast the project has been built! Thanks!

But sorry, the installation problems are still going on. :( In the next step, for installing mxnet for python, the following errors occured:

(base) C:\Users\adeet\git-desktop\incubator-mxnet>pip install --user -e ./python
Obtaining file:///C:/Users/adeet/git-desktop/incubator-mxnet/python
    ERROR: Command errored out with exit status 1:
     command: 'C:\Users\adeet\anaconda3\python.exe' -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'C:\\Users\\adeet\\git-desktop\\incubator-mxnet\\python\\setup.py'"'"'; __file__='"'"'C:\\Users\\adeet\\git-desktop\\incubator-mxnet\\python\\setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info --egg-base 'C:\Users\adeet\AppData\Local\Temp\pip-pip-egg-info-ehf1rqbh'
         cwd: C:\Users\adeet\git-desktop\incubator-mxnet\python\
    Complete output (14 lines):
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "C:\Users\adeet\git-desktop\incubator-mxnet\python\setup.py", line 47, in <module>
        LIB_PATH = libinfo['find_lib_path']()
      File "C:\Users\adeet\git-desktop\incubator-mxnet\python\mxnet/libinfo.py", line 72, in find_lib_path
        raise RuntimeError('Cannot find the MXNet library.\n' +
    RuntimeError: Cannot find the MXNet library.
    List of candidates:
    C:\Users\adeet\git-desktop\incubator-mxnet\python\mxnet\libmxnet.dll
    C:\Users\adeet\git-desktop\incubator-mxnet\python\mxnet\../../lib/libmxnet.dll
    C:\Users\adeet\git-desktop\incubator-mxnet\python\mxnet\../../build/libmxnet.dll
    C:\Users\adeet\git-desktop\incubator-mxnet\python\mxnet\../../build\libmxnet.dll
    C:\Users\adeet\git-desktop\incubator-mxnet\python\mxnet\../../build\Release\libmxnet.dll
    C:\Users\adeet\git-desktop\incubator-mxnet\python\mxnet\../../windows/x64\Release\libmxnet.dll
    ----------------------------------------
ERROR: Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.

The output from the build is below:

(base) C:\Users\adeet\git-desktop\incubator-mxnet\build>cmake ..
-- Building for: Visual Studio 16 2019
-- The C compiler identification is MSVC 19.27.29111.0
-- The CXX compiler identification is MSVC 19.27.29111.0
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: C:/Program Files (x86)/Microsoft Visual Studio/2019/Community/VC/Tools/MSVC/14.27.29110/bin/Hostx64/x64/cl.exe - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: C:/Program Files (x86)/Microsoft Visual Studio/2019/Community/VC/Tools/MSVC/14.27.29110/bin/Hostx64/x64/cl.exe - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- CMAKE_CROSSCOMPILING FALSE
-- CMAKE_HOST_SYSTEM_PROCESSOR AMD64
-- CMAKE_SYSTEM_PROCESSOR AMD64
-- CMAKE_SYSTEM_NAME Windows
-- CMake version '3.18.2' using generator 'Visual Studio 16 2019'
-- The ASM_MASM compiler identification is MSVC
-- Found assembler: C:/Program Files (x86)/Microsoft Visual Studio/2019/Community/VC/Tools/MSVC/14.27.29110/bin/Hostx64/x64/ml64.exe
-- CMAKE_BUILD_TYPE is unset, defaulting to Release
-- Intel MKL-DNN compat: set DNNL_BUILD_EXAMPLES to MKLDNN_BUILD_EXAMPLES with value `OFF`
-- Intel MKL-DNN compat: set DNNL_BUILD_TESTS to MKLDNN_BUILD_TESTS with value `OFF`
-- Intel MKL-DNN compat: set DNNL_ENABLE_JIT_PROFILING to MKLDNN_ENABLE_JIT_PROFILING with value `OFF`
-- Intel MKL-DNN compat: set DNNL_LIBRARY_TYPE to MKLDNN_LIBRARY_TYPE with value `STATIC`
-- Intel MKL-DNN compat: set DNNL_ARCH_OPT_FLAGS to MKLDNN_ARCH_OPT_FLAGS with value ``
-- Looking for pthread.h
-- Looking for pthread.h - not found
-- Found Threads: TRUE
-- Found OpenMP_C: -openmp (found version "2.0")
-- Found OpenMP_CXX: -openmp (found version "2.0")
-- Found OpenMP: TRUE (found version "2.0")
-- GPU support is disabled
-- Could NOT find Doxygen (missing: DOXYGEN_EXECUTABLE)
-- Found Git: C:/Program Files/Git/cmd/git.exe (found version "2.28.0.windows.1")
-- Intel(R) VTune(TM) Amplifier JIT profiling disabled
-- OpenCV ARCH: x64
-- OpenCV RUNTIME: vc15
-- OpenCV STATIC: OFF
-- Found OpenCV: C:/tools/opencv/build (found version "4.1.2") found components: core highgui imgproc imgcodecs
-- Found OpenCV 4.1.2 in C:/tools/opencv/build/x64/vc15/lib
-- You might need to add C:\tools\opencv\build\x64\vc15\bin to your PATH to be able to run your applications.
-- OpenCV 4.1.2 found (C:/tools/opencv/build/x64/vc15/lib)
--  OpenCV_LIBS=opencv_core;opencv_highgui;opencv_imgproc;opencv_imgcodecs
USE_LAPACK is ON
CMake Warning at 3rdparty/googletest/googletest/CMakeLists.txt:47 (project):
  VERSION keyword not followed by a value or was followed by a value that
  expanded to nothing.

-- Found PythonInterp: C:/Users/adeet/anaconda3/python.exe (found version "3.8.3")
-- Found GTest: gtest
-- Looking for clock_gettime in rt
-- Looking for clock_gettime in rt - not found
-- Looking for fopen64
-- Looking for fopen64 - not found
-- Looking for C++ include cxxabi.h
-- Looking for C++ include cxxabi.h - not found
-- Looking for nanosleep
-- Looking for nanosleep - not found
-- Looking for backtrace
-- Looking for backtrace - not found
-- Could NOT find Backtrace (missing: Backtrace_LIBRARY Backtrace_INCLUDE_DIR)
-- Check if the system is big endian
-- Searching 16 bit integer
-- Looking for sys/types.h
-- Looking for sys/types.h - found
-- Looking for stdint.h
-- Looking for stdint.h - found
-- Looking for stddef.h
-- Looking for stddef.h - found
-- Check size of unsigned short
-- Check size of unsigned short - done
-- Searching 16 bit integer - Using unsigned short
-- Check if the system is big endian - little endian
-- C:/Users/adeet/git-desktop/incubator-mxnet/3rdparty/dmlc-core/cmake/build_config.h.in -> include/dmlc/build_config.h
-- Found Python3: C:/Users/adeet/anaconda3/python.exe (found version "3.8.3") found components: Interpreter
-- Google Test not found
-- Configuring done
-- Generating done
-- Build files have been written to: C:/Users/adeet/git-desktop/incubator-mxnet/build

What am I doing wrong?

leezu commented 4 years ago

You haven't run the build. You only configured the build. You still need to run this step (while inside the build directory):

https://github.com/apache/incubator-mxnet/blob/f2e90a22c7190e86441a415b1dbdb47ad3603f30/config/linux.cmake#L31

I'm not sure if Windows will auto-detect the number of CPU cores you have. You can also specify cmake --build . -j NUMCPU where NUMCPU would be the number of parallel workers you'd like to have your build

ekdnam commented 4 years ago

Oh yeah my bad. Thanks.

i did it, and a recurrent error while building was

C:\Users\adeet\git-desktop\incubator-mxnet\3rdparty\mshadow\mshadow\./base.h(173,12): fatal error C1083: Cannot open include file: 'mkl_blas.h': No such file or directory
(compiling source file C:\Users\adeet\git-desktop\incubator-mxnet\{filename}) [C:\Users\adeet\git-desktop\incubator-mxnet\new_build\mxnet.vcxproj]

Later while installing for python (pip install --user -e ./python), the error "libmxnet.dll" not found re-occurred.

szha commented 4 years ago

looks like the compilation failed because of a missing header:

Cannot open include file: 'mkl_blas.h': No such file or directory

I think you may need to either switch to a different blas library or install MKL. @yajiedesign what do you recommend?

yajiedesign commented 4 years ago

please do not change CMakeLists.txt Unless you know what you're doing. if you want change some option use -D param,not change file. I very recommend use cmake-gui in windows.

yajiedesign commented 4 years ago

add -DUSE_MKL_IF_AVAILABLE=0 param force close mkl.

ekdnam commented 4 years ago

looks like the compilation failed because of a missing header:

@szha yes. The requirement of the file is generated from 3rdparty\mshadow\mshadow\base.h.

ekdnam commented 4 years ago

@yajiedesign yeah. I am using cmake-gui, since the beginning, and now am also modifying the values in the config.cmake file itself, so that USE_MKL_IF_AVAILABLE is off. But it is still not working. The same error "libmxnet.dll" is not found is occurring.

ekdnam commented 4 years ago

Do Windows users usually have problems while building and installing, or is this issue just limited to me?

yajiedesign commented 4 years ago

You need compile libmxnet.dll first.To do this, you need to resolve all compilation errors.Before that, you don't have to repeatedly try to execute "PIP install -- user - e . /Python",it certainly won't work.You need to make sure“ libmxnet.dll" Has been compiled,

yajiedesign commented 4 years ago

You can post compilation errors here. Generally speaking, they are caused by configuration errors.

ekdnam commented 4 years ago

Okay.

The command I am using is:

cmake .. -DOpenBLAS_INCLUDE_DIR=C:\opt\OpenBLAS\include\openblas -DUSE_CUDA=0 -DUSE_CUDNN=0 -DUSE_BLAS=open -DUSE_MKL_IF_AVAILABLE=0 -DUSE_MKLDNN=0 -DUSE_LAPACK=0 -DUSE_CPP_PACKAGE=1

The error now is:

np_multinomial_op.cc.obj : error LNK2019: unresolved external symbol "void __cdecl
mxnet::op::CheckPvalGPU<float>(struct mxnet::OpContext const &,float *,int)" (??$Ch
eckPvalGPU@M@op@mxnet@@YAXAEBUOpContext@1@PEAMH@Z) referenced in function "void __c
decl mxnet::op::NumpyMultinomialForward<struct mshadow::cpu>(struct nnvm::NodeAttrs
 const &,struct mxnet::OpContext const &,class std::vector<class mxnet::TBlob,class
 std::allocator<class mxnet::TBlob> > const &,class std::vector<enum mxnet::OpReqTy
pe,class std::allocator<enum mxnet::OpReqType> > const &,class std::vector<class mx
net::TBlob,class std::allocator<class mxnet::TBlob> > const &)" (??$NumpyMultinomia
lForward@Ucpu@mshadow@@@op@mxnet@@YAXAEBUNodeAttrs@nnvm@@AEBUOpContext@1@AEBV?$vect
or@VTBlob@mxnet@@V?$allocator@VTBlob@mxnet@@@std@@@std@@AEBV?$vector@W4OpReqType@mx
net@@V?$allocator@W4OpReqType@mxnet@@@std@@@6@2@Z) [C:\Users\adeet\git-desktop\fina
l-try\incubator-mxnet\build\mxnet.vcxproj]
np_multinomial_op.cc.obj : error LNK2019: unresolved external symbol "void __cdecl
mxnet::op::CheckPvalGPU<double>(struct mxnet::OpContext const &,double *,int)" (??$
CheckPvalGPU@N@op@mxnet@@YAXAEBUOpContext@1@PEANH@Z) referenced in function "void _
_cdecl mxnet::op::NumpyMultinomialForward<struct mshadow::cpu>(struct nnvm::NodeAtt
rs const &,struct mxnet::OpContext const &,class std::vector<class mxnet::TBlob,cla
ss std::allocator<class mxnet::TBlob> > const &,class std::vector<enum mxnet::OpReq
Type,class std::allocator<enum mxnet::OpReqType> > const &,class std::vector<class
mxnet::TBlob,class std::allocator<class mxnet::TBlob> > const &)" (??$NumpyMultinom
ialForward@Ucpu@mshadow@@@op@mxnet@@YAXAEBUNodeAttrs@nnvm@@AEBUOpContext@1@AEBV?$ve
ctor@VTBlob@mxnet@@V?$allocator@VTBlob@mxnet@@@std@@@std@@AEBV?$vector@W4OpReqType@
mxnet@@V?$allocator@W4OpReqType@mxnet@@@std@@@6@2@Z) [C:\Users\adeet\git-desktop\fi
nal-try\incubator-mxnet\build\mxnet.vcxproj]
np_multinomial_op.cc.obj : error LNK2019: unresolved external symbol "void __cdecl
mxnet::op::CheckPvalGPU<class mshadow::half::half_t>(struct mxnet::OpContext const
&,class mshadow::half::half_t *,int)" (??$CheckPvalGPU@Vhalf_t@half@mshadow@@@op@mx
net@@YAXAEBUOpContext@1@PEAVhalf_t@half@mshadow@@H@Z) referenced in function "void
__cdecl mxnet::op::NumpyMultinomialForward<struct mshadow::cpu>(struct nnvm::NodeAt
trs const &,struct mxnet::OpContext const &,class std::vector<class mxnet::TBlob,cl
ass std::allocator<class mxnet::TBlob> > const &,class std::vector<enum mxnet::OpRe
qType,class std::allocator<enum mxnet::OpReqType> > const &,class std::vector<class
 mxnet::TBlob,class std::allocator<class mxnet::TBlob> > const &)" (??$NumpyMultino
mialForward@Ucpu@mshadow@@@op@mxnet@@YAXAEBUNodeAttrs@nnvm@@AEBUOpContext@1@AEBV?$v
ector@VTBlob@mxnet@@V?$allocator@VTBlob@mxnet@@@std@@@std@@AEBV?$vector@W4OpReqType
@mxnet@@V?$allocator@W4OpReqType@mxnet@@@std@@@6@2@Z) [C:\Users\adeet\git-desktop\f
inal-try\incubator-mxnet\build\mxnet.vcxproj]
np_multinomial_op.cc.obj : error LNK2019: unresolved external symbol "void __cdecl
mxnet::op::CheckPvalGPU<class mshadow::bfloat::bf16_t>(struct mxnet::OpContext cons
t &,class mshadow::bfloat::bf16_t *,int)" (??$CheckPvalGPU@Vbf16_t@bfloat@mshadow@@
@op@mxnet@@YAXAEBUOpContext@1@PEAVbf16_t@bfloat@mshadow@@H@Z) referenced in functio
n "void __cdecl mxnet::op::NumpyMultinomialForward<struct mshadow::cpu>(struct nnvm
::NodeAttrs const &,struct mxnet::OpContext const &,class std::vector<class mxnet::
TBlob,class std::allocator<class mxnet::TBlob> > const &,class std::vector<enum mxn
et::OpReqType,class std::allocator<enum mxnet::OpReqType> > const &,class std::vect
or<class mxnet::TBlob,class std::allocator<class mxnet::TBlob> > const &)" (??$Nump
yMultinomialForward@Ucpu@mshadow@@@op@mxnet@@YAXAEBUNodeAttrs@nnvm@@AEBUOpContext@1
@AEBV?$vector@VTBlob@mxnet@@V?$allocator@VTBlob@mxnet@@@std@@@std@@AEBV?$vector@W4O
pReqType@mxnet@@V?$allocator@W4OpReqType@mxnet@@@std@@@6@2@Z) [C:\Users\adeet\git-d
esktop\final-try\incubator-mxnet\build\mxnet.vcxproj]
np_multinomial_op.cc.obj : error LNK2019: unresolved external symbol "void __cdecl
mxnet::op::CheckPvalGPU<unsigned char>(struct mxnet::OpContext const &,unsigned cha
r *,int)" (??$CheckPvalGPU@E@op@mxnet@@YAXAEBUOpContext@1@PEAEH@Z) referenced in fu
nction "void __cdecl mxnet::op::NumpyMultinomialForward<struct mshadow::cpu>(struct
 nnvm::NodeAttrs const &,struct mxnet::OpContext const &,class std::vector<class mx
net::TBlob,class std::allocator<class mxnet::TBlob> > const &,class std::vector<enu
m mxnet::OpReqType,class std::allocator<enum mxnet::OpReqType> > const &,class std:
:vector<class mxnet::TBlob,class std::allocator<class mxnet::TBlob> > const &)" (??
$NumpyMultinomialForward@Ucpu@mshadow@@@op@mxnet@@YAXAEBUNodeAttrs@nnvm@@AEBUOpCont
ext@1@AEBV?$vector@VTBlob@mxnet@@V?$allocator@VTBlob@mxnet@@@std@@@std@@AEBV?$vecto
r@W4OpReqType@mxnet@@V?$allocator@W4OpReqType@mxnet@@@std@@@6@2@Z) [C:\Users\adeet\
git-desktop\final-try\incubator-mxnet\build\mxnet.vcxproj]
np_multinomial_op.cc.obj : error LNK2019: unresolved external symbol "void __cdecl
mxnet::op::CheckPvalGPU<signed char>(struct mxnet::OpContext const &,signed char *,
int)" (??$CheckPvalGPU@C@op@mxnet@@YAXAEBUOpContext@1@PEACH@Z) referenced in functi
on "void __cdecl mxnet::op::NumpyMultinomialForward<struct mshadow::cpu>(struct nnv
m::NodeAttrs const &,struct mxnet::OpContext const &,class std::vector<class mxnet:
:TBlob,class std::allocator<class mxnet::TBlob> > const &,class std::vector<enum mx
net::OpReqType,class std::allocator<enum mxnet::OpReqType> > const &,class std::vec
tor<class mxnet::TBlob,class std::allocator<class mxnet::TBlob> > const &)" (??$Num
pyMultinomialForward@Ucpu@mshadow@@@op@mxnet@@YAXAEBUNodeAttrs@nnvm@@AEBUOpContext@
1@AEBV?$vector@VTBlob@mxnet@@V?$allocator@VTBlob@mxnet@@@std@@@std@@AEBV?$vector@W4
OpReqType@mxnet@@V?$allocator@W4OpReqType@mxnet@@@std@@@6@2@Z) [C:\Users\adeet\git-
desktop\final-try\incubator-mxnet\build\mxnet.vcxproj]
np_multinomial_op.cc.obj : error LNK2019: unresolved external symbol "void __cdecl
mxnet::op::CheckPvalGPU<int>(struct mxnet::OpContext const &,int *,int)" (??$CheckP
valGPU@H@op@mxnet@@YAXAEBUOpContext@1@PEAHH@Z) referenced in function "void __cdecl
 mxnet::op::NumpyMultinomialForward<struct mshadow::cpu>(struct nnvm::NodeAttrs con
st &,struct mxnet::OpContext const &,class std::vector<class mxnet::TBlob,class std
::allocator<class mxnet::TBlob> > const &,class std::vector<enum mxnet::OpReqType,c
lass std::allocator<enum mxnet::OpReqType> > const &,class std::vector<class mxnet:
:TBlob,class std::allocator<class mxnet::TBlob> > const &)" (??$NumpyMultinomialFor
ward@Ucpu@mshadow@@@op@mxnet@@YAXAEBUNodeAttrs@nnvm@@AEBUOpContext@1@AEBV?$vector@V
TBlob@mxnet@@V?$allocator@VTBlob@mxnet@@@std@@@std@@AEBV?$vector@W4OpReqType@mxnet@
@V?$allocator@W4OpReqType@mxnet@@@std@@@6@2@Z) [C:\Users\adeet\git-desktop\final-tr
y\incubator-mxnet\build\mxnet.vcxproj]
np_multinomial_op.cc.obj : error LNK2019: unresolved external symbol "void __cdecl
mxnet::op::CheckPvalGPU<__int64>(struct mxnet::OpContext const &,__int64 *,int)" (?
?$CheckPvalGPU@_J@op@mxnet@@YAXAEBUOpContext@1@PEA_JH@Z) referenced in function "vo
id __cdecl mxnet::op::NumpyMultinomialForward<struct mshadow::cpu>(struct nnvm::Nod
eAttrs const &,struct mxnet::OpContext const &,class std::vector<class mxnet::TBlob
,class std::allocator<class mxnet::TBlob> > const &,class std::vector<enum mxnet::O
pReqType,class std::allocator<enum mxnet::OpReqType> > const &,class std::vector<cl
ass mxnet::TBlob,class std::allocator<class mxnet::TBlob> > const &)" (??$NumpyMult
inomialForward@Ucpu@mshadow@@@op@mxnet@@YAXAEBUNodeAttrs@nnvm@@AEBUOpContext@1@AEBV
?$vector@VTBlob@mxnet@@V?$allocator@VTBlob@mxnet@@@std@@@std@@AEBV?$vector@W4OpReqT
ype@mxnet@@V?$allocator@W4OpReqType@mxnet@@@std@@@6@2@Z) [C:\Users\adeet\git-deskto
p\final-try\incubator-mxnet\build\mxnet.vcxproj]
C:\Users\adeet\git-desktop\final-try\incubator-mxnet\build\Debug\libmxnet.dll : fat
al error LNK1120: 8 unresolved externals [C:\Users\adeet\git-desktop\final-try\incu
bator-mxnet\build\mxnet.vcxproj]
ekdnam commented 4 years ago

I was trying out various different configurations a good part of yesterday.

I found out that the cblas.h and blas.h error(s) disappeared when -DOpenBLAS_INCLUDE_DIR=C:\opt\OpenBLAS\include\openblas and -DUSE_MKL_IF_AVAILABLE=0 was added.

A suggestion, it should be specified in cmake\modules\FindOpenBLAS.cmake where OpenBLAS_INCLUDE_DIR should point to.

Because earlier, I had set it to c:\opt\OpenBLAS\include, trying to emulate the way followed here

https://github.com/apache/incubator-mxnet/blob/363caacf6f5ab2a1c22e8bf730e6eca3302a926f/cmake/Modules/FindOpenBLAS.cmake#L28-L29

and then the cblas.h error was occuring. The cblas.h file is located @ C:\opt\OpenBLAS\include\openblas.

Maybe I interpreted it in a wrong way, don't know.

yajiedesign commented 4 years ago

missing OpenBLAS_LIB=path_to_open_blas_lib

yajiedesign commented 4 years ago

not set -DUSE_LAPACK=0 not set -DUSE_CPP_PACKAGE=1

yajiedesign commented 4 years ago

do you run git submodule init and git submodule update.

ekdnam commented 4 years ago

missing OpenBLAS_LIB=path_to_open_blas_lib

The cmake gui shows the correct path for OpenBLAS_LIB, so I have not specified it. If you say so, I will add it to the cmake command.

ekdnam commented 4 years ago

not set -DUSE_LAPACK=0 not set -DUSE_CPP_PACKAGE=1

Okay. Will not add them to the cmake command. Was adding them as while I was running the cmake .. command, an error/warning was that LAPACK not found.

ekdnam commented 4 years ago

do you run git submodule init and git submodule update.

I don't know about these commands. So have not run them

ekdnam commented 4 years ago

The cmake command now is

cmake .. -DOpenBLAS_INCLUDE_DIR=C:\opt\OpenBLAS\include\openblas -DOpenBLAS_LIB=C:\opt\OpenBLAS\lib\openblas.lib -DUSE_CUDA=0 -DUSE_CUDNN=0 -DUSE_BLAS=open -DUSE_MKL_IF_AVAILABLE=0 -DUSE_MKLDNN=0

After building the project, only one error was there

LINK : fatal error LNK1104: cannot open file 'flangmain.lib' [C:\Users\adeet\git-desktop\final-try\incubator-mxnet\build\mxnet.vcxproj]
leezu commented 4 years ago

The cmake gui shows the correct path for OpenBLAS_LIB, so I have not specified it. If you say so, I will add it to the cmake command.

The problem is that you should not run cmake multiple times without first deleting the CMakeCache.txt. Some of your errors can be related to having an inconsistent set of options from prior configuration runs and the newly specified options.

yajiedesign commented 4 years ago

you need run "git submodule init" and "git submodule update" pull 3rdparty and submodule code.

yajiedesign commented 4 years ago

look like you openblas is static link.change to dynamic link.

ekdnam commented 4 years ago

The problem is that you should not run cmake multiple times without first deleting the CMakeCache.txt. Some of your errors can be related to having an inconsistent set of options from prior configuration runs and the newly specified options.

I remove CMakeCache.txt. everytime while building. Perhaps it solves the problem you mention.

ekdnam commented 4 years ago

you need run "git submodule init" and "git submodule update" pull 3rdparty and submodule code.

I have cloned the repo recursively. So imo there is no need to do that.

ekdnam commented 4 years ago

look like you openblas is static link.change to dynamic link.

How to do this?

yajiedesign commented 4 years ago

download openblas official and set -DOpenBLAS_LIB=xxxx\libopenblas.dll.a

ekdnam commented 4 years ago

I think there is some problem with my laptop leading to an inability to build MXNet. I have followed the entirety of the instructions we talked about here, yet no result :((((