Difficulties Building on Windows

[J-norton]

I'm using Windows 10 to build and have been trying to build Caffe 2 using Visual Studio 14 2015 for a while with no luck. I've built protoc, using the directions described in the installation and pointed CMake to the location for that executable.

At that point I used the CMake GUI to configure the solution generation and I generated the solutions. I I end up with a solution with 16 projects (I disabled a lot of optional items). When I try to build the solution, I get an error that an #include directive can't be resolved.

C:\Users\James\Source\Repos\caffe2\caffe2/operators/funhash_op.h(4): fatal error C1083: Cannot open include file: 'xxhash.h': No such file or directory

My guess is that something in the CMake configuration is wrong, but I'm not sure what. I am attaching the CMakeCache.txt file that is generated. It's worth noting that the error that I see is within the Caffe2_CPU project. I have an Nvidia GPU and CUDA 8.0 installed, so if there is a way to disable the CPU project from the build that would be great.

Any help is much appreciated.

CMakeCache.txt

[J-norton]

Has anyone else encountered this issue before? I merged the latest and spent a while trying to configure different CMake options, but I am consistently seeing this error that xxhash.h is not found regardless of what I try.

Anybody built this on Windows before willing to share their experience?

[Yangqing]

This seems to happen because of funhash.h - it is currently being moved to experiments/ and is not being compiled in default, could you pull the latest repository and retry?

[cgpoh]

@J-norton Below are the steps that I follow to build successfully on Windows

• Follow the steps on caffe2 installation but with some modifications.

• Initially, I tried going the regular python install way but failed to get caffe2 working and so I removed my python installation and installed Anaconda 2 instead.

• After installing Anaconda 2, follow the Anaconda install but install more packages by using the following command: conda install -y --channel https://conda.anaconda.org/conda-forge graphviz hypothesis numpy pydot-ng python-lmdb requests zeromq future gflags protobuf=3.2.0

• For leveldb, modified the leveldb visual studio project to disable Boost auto linking by defining the preprocessor symbol BOOST_ALL_NO_LIB and add the following libraries (your version may be different from mine) in Librarian->General->Additional Dependencies libboost_date_time-vc140-mt-s-1_59.lib libboost_filesystem-vc140-mt-s-1_59.lib libboost_system-vc140-mt-s-1_59.lib

• I have to modify FindSnappy.cmake and FindGFlags.cmake in order for CMake to work and also add GFLAGS_ROOT_DIR and SNAPPY_ROOT_DIR to Windows environment variables. FindSnappy.cmake modification:

  
  # Find the Snappy libraries
  #
  # The following variables are optionally searched for defaults
  #  SNAPPY_ROOT_DIR:    Base directory where all Snappy components are found
  #
  # The following are set after configuration is done:
  #  SNAPPY_FOUND
  #  Snappy_INCLUDE_DIR
  #  Snappy_LIBRARIES

find_path(Snappy_INCLUDE_DIR NAMES snappy.h PATHS $ENV{SNAPPY_ROOT_DIR} $ENV{SNAPPY_ROOT_DIR}/include)

find_library(Snappy_LIBRARIES NAMES snappy64 PATHS $ENV{SNAPPY_ROOT_DIR} $ENV{SNAPPY_ROOT_DIR}/lib)

include(FindPackageHandleStandardArgs) find_package_handle_standard_args(Snappy DEFAULT_MSG Snappy_INCLUDE_DIR Snappy_LIBRARIES)

if(SNAPPY_FOUND) message(STATUS "Found Snappy (include: ${Snappy_INCLUDE_DIR}, library: ${Snappy_LIBRARIES})") mark_as_advanced(Snappy_INCLUDE_DIR Snappy_LIBRARIES) caffe_parse_header(${Snappy_INCLUDE_DIR}/snappy-stubs-public.h SNAPPY_VERION_LINES SNAPPY_MAJOR SNAPPY_MINOR SNAPPY_PATCHLEVEL) set(Snappy_VERSION "${SNAPPY_MAJOR}.${SNAPPY_MINOR}.${SNAPPY_PATCHLEVEL}") endif()

and FindGFlags.cmake modification:

- Try to find GFLAGS

#

The following variables are optionally searched for defaults

GFLAGS_ROOT_DIR: Base directory where all GFLAGS components are found

#

The following are set after configuration is done:

GFLAGS_FOUND

GFLAGS_INCLUDE_DIRS

GFLAGS_LIBRARIES

GFLAGS_LIBRARYRARY_DIRS

include(FindPackageHandleStandardArgs)

set(GFLAGS_ROOT_DIR "" CACHE PATH "Folder contains Gflags")

We are testing only a couple of files in the include directories

if(WIN32) find_path(GFLAGS_INCLUDE_DIR gflags/gflags.h PATHS $ENV{GFLAGS_ROOT_DIR}) else() find_path(GFLAGS_INCLUDE_DIR gflags/gflags.h PATHS $ENV{GFLAGS_ROOT_DIR}) endif()

if(MSVC) find_library(GFLAGS_LIBRARY_RELEASE NAMES gflags PATHS $ENV{GFLAGS_ROOT_DIR} PATH_SUFFIXES Lib)

find_library(GFLAGS_LIBRARY_DEBUG
    NAMES gflagsd
    PATHS $ENV{GFLAGS_ROOT_DIR}
    PATH_SUFFIXES Lib)

set(GFLAGS_LIBRARY optimized ${GFLAGS_LIBRARY_RELEASE} debug ${GFLAGS_LIBRARY_DEBUG})

else() find_library(GFLAGS_LIBRARY gflags) endif()

find_package_handle_standard_args(GFlags DEFAULT_MSG GFLAGS_INCLUDE_DIR GFLAGS_LIBRARY)

if(GFLAGS_FOUND) set(GFLAGS_INCLUDE_DIRS ${GFLAGS_INCLUDE_DIR}) set(GFLAGS_LIBRARIES ${GFLAGS_LIBRARY}) message(STATUS "Found gflags (include: ${GFLAGS_INCLUDE_DIR}, library: ${GFLAGS_LIBRARY})") mark_as_advanced(GFLAGS_LIBRARY_DEBUG GFLAGS_LIBRARY_RELEASE GFLAGS_LIBRARY GFLAGS_INCLUDE_DIR GFLAGS_ROOT_DIR) endif()

- Run `build_windows.bat` and once it build successfully, remember to set `PYTHONPATH` to your caffe2 build directory in the Windows environment variable.

I hope it helps :)

[J-norton]

Oh cool thanks for sharing! I had given up hope and was happily coding with Caffe 1 XD.

I just pulled the latest and I'll try to rebuild, if that doesn't work I'll try your suggestions @cgpoh and see if that fixes it.

Update:

Okay so pulling the latest changes did not work (I got like 3000 link errors), @cgpoh I'm following your guide now, thanks for the detailed instructions. I'm hunting down the build dependencies. Mine are slightly different from yours, but your script modification for gflags worked for me.

I'm a bit confused by your advice for LevelDB. I see the following error in CMake GUI:

CMake Warning at cmake/Dependencies.cmake:126 (message): Not compiling with LevelDB. Suppress this warning with -DUSE_LEVELDB=OFF Call Stack (most recent call first): CMakeLists.txt:72 (include)

It looks like CMake expects to find LevelDB, but you mention rebuilding it in Visual Studio, do I need to generate the project, build it, and then set the paths? Would you mind attaching the CMakeCache.txt file that you have so that I can compare it against my own?

Also in general, I'm a bit confused as to why I need to satisfy the dependencies in the first place to build the core Caffe 2. The CMake scripts seem to think that building without the 3rd party libraries is possible (glog, gflags, etc.)

[cgpoh]

Hi @J-norton, I'm attaching my CMakeCache.txt CMakeCache.txt

After my successful build in Windows, I'm facing a problem with make_mnist_db.exe. I didn't managed to generate db with this exe. Is it possible to let me know if you are facing this problem too?

[J-norton]

Hey thanks for sharing. Yeah I'll keep you posted when I take a look. I'm currently using Caffe 1 for work but we're considering upgrading, anyways I haven't had a ton of time to invest in Caffe 2 but I'm going to take a look tonight.

Are you talking about the digit recognition example network? I think I encountered the same problem when using Caffe 1, if that's the problem you're talking about.

[Isaamarod]

@cgpoh I have the same problem with make_mnist_db.exe