when compile Caffe2, build_windows.bat flashing and exit

[Kasdision]

When I try to run build_windows.bat, I change" -DBUILD_PYTHON=OFF^" to " -DBUILD_PYTHON=ON^", and then the bat is flashing and exit. Does anyone else has this problem? My computer enviroment is: win10/VS2015/python2.7.14/cuda9.0/cudnn7.1 Please help me, any assistance you could give will be greatly appreciated!

CMake summary output

CAFFE2_ROOT=E:\caffe2\scripts\..
CMAKE_GENERATOR="Visual Studio 14 2015 Win64"
CMAKE_BUILD_TYPE=Release
-- Selecting Windows SDK version  to target Windows 10.0.16299.
-- Need to define long as a separate typeid.
-- std::exception_ptr is supported.
-- NUMA is not available
-- Current compiler supports avx2 extention. Will build perfkernels.
CMake Warning at cmake/public/protobuf.cmake:79 (message):
  Protobuf cannot be found.  Depending on whether you are building Caffe2 or
  a Caffe2 dependent library, the next warning / error will give you more
  info.
Call Stack (most recent call first):
  cmake/ProtoBuf.cmake:53 (include)
  cmake/Dependencies.cmake:2 (include)
  CMakeLists.txt:101 (include)

CMake Warning at cmake/ProtoBuf.cmake:57 (message):
  Protobuf cannot be found.  Caffe2 will automatically switch to use own
  protobuf under third_party.  Note that this behavior may change in the
  future, and you will need to specify -DBUILD_CUSTOM_PROTOBUF=ON explicitly.
Call Stack (most recent call first):
  cmake/Dependencies.cmake:2 (include)
  CMakeLists.txt:101 (include)

-- Use custom protobuf build.
-- Caffe2 protobuf include directory: $<BUILD_INTERFACE:E:/caffe2/third_party/protobuf/src>$<INSTALL_INTERFACE:include>
-- The BLAS backend of choice:Eigen
CMake Warning at cmake/Dependencies.cmake:183 (message):
  NUMA is currently only supported under Linux.
Call Stack (most recent call first):
  CMakeLists.txt:101 (include)

CMake Warning at cmake/Dependencies.cmake:256 (find_package):
  By not providing "FindEigen3.cmake" in CMAKE_MODULE_PATH this project has
  asked CMake to find a package configuration file provided by "Eigen3", but
  CMake did not find one.

  Could not find a package configuration file provided by "Eigen3" with any
  of the following names:

    Eigen3Config.cmake
    eigen3-config.cmake

  Add the installation prefix of "Eigen3" to CMAKE_PREFIX_PATH or set
  "Eigen3_DIR" to a directory containing one of the above files.  If "Eigen3"
  provides a separate development package or SDK, be sure it has been
  installed.
Call Stack (most recent call first):
  CMakeLists.txt:101 (include)

-- Did not find system Eigen. Using third party subdirectory.
-- NumPy ver. 1.14.1 found (include: C:/Python27/lib/site-packages/numpy/core/include)
-- Could NOT find MPI_C (missing: MPI_C_LIB_NAMES MPI_C_HEADER_DIR MPI_C_WORKS)
-- Could NOT find MPI_CXX (missing: MPI_CXX_LIB_NAMES MPI_CXX_HEADER_DIR MPI_CXX_WORKS)
-- Could NOT find MPI (missing: MPI_C_FOUND MPI_CXX_FOUND)
CMake Warning at cmake/Dependencies.cmake:315 (message):
  Not compiling with MPI.  Suppress this warning with -DUSE_MPI=OFF
Call Stack (most recent call first):
  CMakeLists.txt:101 (include)

-- Caffe2: CUDA detected: 9.0
-- Found cuDNN: v7.1.1  (include: C:/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v9.0/include, library: C:/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v9.0/lib/x64/cudnn.lib)
-- Automatic GPU detection failed. Building for all known architectures.
-- Added CUDA NVCC flags for: sm_30 sm_35 sm_50 sm_52 sm_60 sm_61 sm_70
CMake Warning at cmake/Dependencies.cmake:363 (message):
  NCCL is currently only supported under Linux.
Call Stack (most recent call first):
  CMakeLists.txt:101 (include)

-- Could NOT find CUB (missing: CUB_INCLUDE_DIR)
CMake Warning at cmake/Dependencies.cmake:383 (message):
  Gloo can only be used on Linux.
Call Stack (most recent call first):
  CMakeLists.txt:101 (include)

CMake Warning at cmake/Dependencies.cmake:443 (message):
  mobile opengl is only used in android or ios builds.
Call Stack (most recent call first):
  CMakeLists.txt:101 (include)

CMake Warning at cmake/Dependencies.cmake:519 (message):
  Metal is only used in ios builds.
Call Stack (most recent call first):
  CMakeLists.txt:101 (include)

-- NCCL operators skipped due to no CUDA support
-- Excluding image processing operators due to no opencv
-- Excluding video processing operators due to no opencv
-- Excluding mkl operators as we are not using mkl
-- MPI operators skipped due to no MPI support
-- Include Observer library
ERROR Failed to remove indentation from:
"""

      from distutils import sysconfig
      print(sysconfig.get_python_lib(prefix=''))

"""
    Python dedent failed with error code: 系统找不到指定的文件。
CMake Error at cmake/Utils.cmake:177 (message):
   Python dedent failed with error code: 系统找不到指定的文件。
Call Stack (most recent call first):
  cmake/Utils.cmake:200 (dedent)
  caffe2/CMakeLists.txt:146 (pycmd)

-- Configuring incomplete, errors occurred!
See also "E:/caffe2/build/CMakeFiles/CMakeOutput.log".
See also "E:/caffe2/build/CMakeFiles/CMakeError.log".
"Caffe2 building failed"

[Kasdision]

@Erotemic #1982 I see this same problem, could tell me how to solve this problem? ps: I have tried to slove this by what you said in #1982 ,but it can't help

[Erotemic]

Yes, it seems that the newlines in stdout/stdin are actually not causing any issue.

I think the solution is that dedent is giving an uninformative error message.

Did you try what I wrote in this comment?

So, I actually tested this on my windows box, but I was unable to reproduce the error when running via cmd.exe. Do you have python installed? Is python on the path?

What happens if you download this file: https://github.com/Erotemic/misc/blob/master/cmake/utils.cmake

and then run cmake -P utils.cmake?

[Kasdision]

@Erotemic Thank you for your reply, when I run cmake -P utils.cmake, I got error like this:

C:\Users\Administrator>cmake -P utils.cmake
CMake Warning (dev) at utils.cmake:76 (if):
  if given arguments:

    "TRUE"

  An argument named "TRUE" appears in a conditional statement.  Policy
  CMP0012 is not set: if() recognizes numbers and boolean constants.  Run
  "cmake --help-policy CMP0012" for policy details.  Use the cmake_policy
  command to set the policy and suppress this warning.
This warning is for project developers.  Use -Wno-dev to suppress it.

ERROR Failed to remove indentation from:
"""

    from distutils import sysconfig
    print(sysconfig.get_python_lib(prefix=''))

"""
    Python dedent failed with error code: 系统找不到指定的文件。
CMake Error at utils.cmake:31 (message):
   Python dedent failed with error code: 系统找不到指定的文件。
Call Stack (most recent call first):
  utils.cmake:54 (dedent)
  utils.cmake:97 (pycmd)

It seems like the same error to the "build_windows.bat", could I ask what's the problem it is?Is something about my python enviroment wrong?

[Erotemic]

Its very possible that something in your Python environment is wrong.

To remind myself, I'm posting that "系统找不到指定的文件" means: "System can't find the specified file.

In the same command prompt where you ran build_windows.bat can you run:

python -c "import textwrap; print(textwrap.__file__)"

and report the output?

Also change the function dedent in cmake as such:

function(dedent outvar text)
  # Use PYTHON_EXECUTABLE if it is defined, otherwise default to python
  if ("${PYTHON_EXECUTABLE}" STREQUAL "")
    set(_python_exe "python")
  else()
    set(_python_exe "${PYTHON_EXECUTABLE}")
  endif()
  set(_fixup_cmd "import sys; from textwrap import dedent; print(dedent(sys.stdin.read()))")
  # Use echo to pipe the text to python's stdinput. This prevents us from
  # needing to worry about any sort of special escaping.
  message(STATUS " PYTHON_EXECUTABLE: \"${PYTHON_EXECUTABLE}\"")
  message(STATUS " _python_exe: \"${_python_exe}\"")
  message(STATUS " _fixup_cmd: \"${_fixup_cmd}\"")
  message(STATUS " text: \"${text}\"")
  execute_process(
    COMMAND echo "${text}"
    COMMAND "${_python_exe}" -c "${_fixup_cmd}"
    RESULT_VARIABLE _dedent_exitcode
    OUTPUT_VARIABLE _dedent_text)
  if(NOT ${_dedent_exitcode} EQUAL 0)
    message(ERROR " Failed to remove indentation from: \n\"\"\"\n${text}\n\"\"\"
    Python dedent failed with error code: ${_dedent_exitcode}")
    message(FATAL_ERROR " Python dedent failed with error code: ${_dedent_exitcode}")
  endif()
  # Remove supurflous newlines (artifacts of print)
  string(STRIP "${_dedent_text}" _dedent_text)
  set(${outvar} "${_dedent_text}" PARENT_SCOPE)
endfunction()

and then report what the output is?

[Kasdision]

@Erotemic When I run: python -c "import textwrap; print(textwrap.__file__)" The output is:

C:\Users\Administrator>python -c "import textwrap; print(textwrap.__file__)"
C:\Python27\lib\textwrap.pyc

And when I replace the function dedent in cmake that you give, I got that:

C:\Users\Administrator>cmake -P utils.cmake
CMake Warning (dev) at utils.cmake:80 (if):
  if given arguments:

    "TRUE"

  An argument named "TRUE" appears in a conditional statement.  Policy
  CMP0012 is not set: if() recognizes numbers and boolean constants.  Run
  "cmake --help-policy CMP0012" for policy details.  Use the cmake_policy
  command to set the policy and suppress this warning.
This warning is for project developers.  Use -Wno-dev to suppress it.

--  PYTHON_EXECUTABLE: ""
--  _python_exe: "python"
--  _fixup_cmd: "import sys; from textwrap import dedent; print(dedent(sys.stdin.read()))"
--  text: "
    from distutils import sysconfig
    print(sysconfig.get_python_lib(prefix=''))
  "
ERROR Failed to remove indentation from:
"""

    from distutils import sysconfig
    print(sysconfig.get_python_lib(prefix=''))

"""
    Python dedent failed with error code: 系统找不到指定的文件。
CMake Error at utils.cmake:35 (message):
   Python dedent failed with error code: 系统找不到指定的文件。
Call Stack (most recent call first):
  utils.cmake:58 (dedent)
  utils.cmake:101 (pycmd)

What does that mean? I just can not understand...

[Erotemic]

The first test was simply to verify that you had python in your path.

The second test was to show that cmake found python correctly, and sanity checks of the variable values that matter.

Looking at these results I don't see anything out of the ordinary.

What happens if you remove the line

COMMAND "${_python_exe}" -c "${_fixup_cmd}", does the command fail on the echo?

Lastly, we should try something simple like

  execute_process(
    COMMAND echo "${text}"
    COMMAND "${_python_exe}" -c "print('hello_world')"
    RESULT_VARIABLE _dedent_exitcode
    OUTPUT_VARIABLE _dedent_text)

and check the values of _dedent_exitcode and _dedent_text

If these don't work does the error persist if you upgrade to Python3? Python2 only has a few years of support left anyway. Python3 may fix the issue if its a unicode problem.

[Kasdision]

@Erotemic I have tried to remove the line: COMMAND "${_python_exe}" -c "${_fixup_cmd}" I got:

C:\Users\Administrator>cmake -P utils.cmake
CMake Warning (dev) at utils.cmake:75 (if):
  if given arguments:

    "TRUE"

  An argument named "TRUE" appears in a conditional statement.  Policy
  CMP0012 is not set: if() recognizes numbers and boolean constants.  Run
  "cmake --help-policy CMP0012" for policy details.  Use the cmake_policy
  command to set the policy and suppress this warning.
This warning is for project developers.  Use -Wno-dev to suppress it.

ERROR Failed to remove indentation from:
"""

    from distutils import sysconfig
    print(sysconfig.get_python_lib(prefix=''))

"""
    Python dedent failed with error code: 系统找不到指定的文件。
CMake Error at utils.cmake:30 (message):
   Python dedent failed with error code: 系统找不到指定的文件。
Call Stack (most recent call first):
  utils.cmake:53 (dedent)
  utils.cmake:96 (pycmd)

It seems the problem still exists..... When I try something you give:

execute_process(
    COMMAND echo "${text}"
    COMMAND "${_python_exe}" -c "print('hello_world')"
    RESULT_VARIABLE _dedent_exitcode
    OUTPUT_VARIABLE _dedent_text)

the output is still:

C:\Users\Administrator>cmake -P utils.cmake
CMake Warning (dev) at utils.cmake:76 (if):
  if given arguments:

    "TRUE"

  An argument named "TRUE" appears in a conditional statement.  Policy
  CMP0012 is not set: if() recognizes numbers and boolean constants.  Run
  "cmake --help-policy CMP0012" for policy details.  Use the cmake_policy
  command to set the policy and suppress this warning.
This warning is for project developers.  Use -Wno-dev to suppress it.

ERROR Failed to remove indentation from:
"""

    from distutils import sysconfig
    print(sysconfig.get_python_lib(prefix=''))

"""
    Python dedent failed with error code: 系统找不到指定的文件。
CMake Error at utils.cmake:31 (message):
   Python dedent failed with error code: 系统找不到指定的文件。
Call Stack (most recent call first):
  utils.cmake:54 (dedent)
  utils.cmake:97 (pycmd)

I find that code is used to remove indentation in python-Lib-sitepackages, is that true? I also try to run this in my python:

    from distutils import sysconfig
    print(sysconfig.get_python_lib(prefix=''))

I got :

>>> from distutils import sysconfig
>>> print(sysconfig.get_python_lib(prefix=''))
Lib\site-packages

And finally I change my python(from 2.7.14 to 3.6.4), it still can't work and it has the same error......now I’m lost myself..... Can this problem be solved? I think it will be many people who have the same problem, if he/she wants to install caffe2 in windows. I'm looking forward to your reply!

[Kasdision]

2228 The problem has been solved!!

[wanghenglily]

@Kasdision could you tell me how to solved the problem? I face the same problem....