Installing DensePose is not an easy thing except building it in a docker container. Here, I provided an installation guide based on the official DensePose Installation and an installation guide provided by @linkinpark213.
mkl to the latest version (2019.3-199) since relevant bug has been fixed.protobuf package.It is recommended to create a python environment with a speficied python version (e.g., 2.7/3.6/3.7) via Conda by this following command,
$ conda create -y -n your_env_name python=x.xNote that instructions like x.x or pythonx.x (below) indicate the specific python version you installed in your created conda environment, which can be 2.7/3.6/3.7 or any other valid python version.
For convenience in the following instructions, we set some environment variables accordingly:
export CONDA_ENV_PATH=/path/to/anaconda3/envs/your_env_name
export CONDA_PKGS_PATH=/path/to/anaconda3/envs/your_env_name/lib/pythonx.x/site-packagesThen, install the following python packages under the created environment:
conda install -y numpy setuptools cffi typing pyyaml=3.13 mkl=2019.1 mkl-include=2019.1conda install -y cython matplotlibconda install -y pydot future networkxconda install -y opencv mock scipy h5pypip install chumpyBy default, conda will install the latest version of these packages. However, package pyyaml with version 4.x may cause a ConstructorError when loading a .yaml file according to this issue, current solution is downgrade pyyaml to version 3.13 (>= 3.12).
# COCOAPI=/path/to/clone/cocoapi
$ git clone https://github.com/cocodataset/cocoapi.git $COCOAPI
$ cd $COCOAPI/PythonAPI
# Install into global site-packages
$ make installNote that instructions like # COCOAPI=/path/to/install/cocoapi indicate that you should pick a path where you'd like to have the software cloned and then set an environment variable (COCOAPI in this case) accordingly.
Caffe2 has been merged and integrated into PyTorch, you can install them either from source code or from binaries via Conda.
See Install PyTorch from Source.
Commands to install from binaries via Conda or pip wheels are on PyTorch website. E.g. (in my case),
$ conda install pytorch torchvision cudatoolkit=9.0 -c pytorchWhen the installation of pytorch & caffe2 is done, the python API package (named torch and caffe2) is installed in $CONDA_PKGS_PATH, for convenience in the following instructions, set some environment variables:
export TORCH_PATH=$CONDA_PKGS_PATH/torch
export CAFFE2_INCLUDE_PATH=$TORCH_PATH/include/caffe2Before installing protobuf, you need to check which version (currently, 3.6.1) is specified in the header of $CAFFE2_INCLUDE_PATH/proto/caffe2.pb.h like the following code,
// Generated by the protocol buffer compiler. DO NOT EDIT!
// source: caffe2/proto/caffe2.proto
#ifndef PROTOBUF_INCLUDED_caffe2_2fproto_2fcaffe2_2eproto
#define PROTOBUF_INCLUDED_caffe2_2fproto_2fcaffe2_2eproto
#include <string>
#include <google/protobuf/stubs/common.h>
#if GOOGLE_PROTOBUF_VERSION < 3006001
#error This file was generated by a newer version of protoc which is
#error incompatible with your Protocol Buffer headers. Please update
#error your headers.
#endif
#if 3006001 < GOOGLE_PROTOBUF_MIN_PROTOC_VERSION
#error This file was generated by an older version of protoc which is
#error incompatible with your Protocol Buffer headers. Please
#error regenerate this file with a newer version of protoc.
#endifThen, install protobuf via conda by conda install protobuf=3.6.1 accordingly.
Adjust your (This issue has been solved by the official Detectron).PYTHONPATH environment variable to include this location: $TORCH_PATH/lib (referred to the issue: Detectron ops lib not found). If this doesn't work when setting up python modules of densepose, try creating a symbol link to this path in the root directory of your densepose: ln -s $TORCH_PATH/lib /path/to/your/densepose/lib
Please ensure that your Caffe2 installation and protobuf installation was successful before proceeding by running the following commands and checking their output as directed in the comments.
# To check if Caffe2 build was successful
$ python -c 'from caffe2.python import core' 2>/dev/null && echo "Success" || echo "Failure"
# To check if Caffe2 GPU build was successful
# This must print a number > 0 in order to use Detectron
$ python -c 'from caffe2.python import workspace; print(workspace.NumCudaDevices())'To avoid this issue in case multiple CUDA libraries have been installed in your system and the symbol link /usr/local/cuda doesn't link to the specific version of CUDA that you're using, you need to replace /usr/local/cuda/lib64/libculibos.a with /usr/local/cuda-x.x/lib64/libculibos.a in $TORCH_PATH/share/cmake/Caffe2/Caffe2Target.cmake, otherwise it may trigger a linking error when installing the custom DensePose operator. (This issue only occurs in which the caffe2 was installed from binaries via conda).
If Caffe2 was installed via Conda, then you need to make sure that you have the feasible GCC compiler in your system before building densepose (referred to this issue).
So far, GCC-4.9.2 has been successfully tested to be used to compile custom operators in densepose which depends on Caffe2 libraries. You can also run the following command to see which versions of GCC are supported in the precompiled libcaffe2.so library like the following output,
$ strings -a $TORCH_PATH/lib/libcaffe2.so | grep "GCC: ("
$ # Output from the command above
$ GCC: (GNU) 4.9.2 20150212 (Red Hat 4.9.2-6)`Download GCC-4.9.2 source code and build it by the following commands,
$ mkdir /path/to/gcc-4.9.2/build && cd /path/to/gcc-4.9.2/build
$ ../configure --prefix=/path/to/build --enable-checking=release --enable-languages=c,c++ --disable-multilib
$ make
$ make install
$ cd bin/
$ ln -s gcc cc # create a symbol link 'cc' for 'gcc'
$ cp -r include/c++ $CONDA_ENV_PATH/includeAfter installing gcc-4.9.2 successfully, add /path/to/gcc-4.9.2/build/bin and /path/to/gcc-4.9.2/build/lib64 to your $PATH and $LD_LIBRARY_PATH environment variables, respectively.
Clone the Densepose repository:
# DENSEPOSE=/path/to/clone/densepose
# official DensePose repo
$ git clone https://github.com/facebookresearch/densepose $DENSEPOSE
# or clone my forked repo with python3 compatibility support
$ git clone https://github.com/Johnqczhang/DensePose $DENSEPOSESet up Python modules:
$ cd $DENSEPOSE
$ python setup.py developCheck that Detectron tests pass (e.g. for SpatialNarrowAsOp test):
$ python $DENSEPOSE/detectron/tests/test_spatial_narrow_as_op.pyEdit $DENSEPOSE/CMakeLists.txt (Thanks to @hyousamk's solution), you can download CMakeLists.txt from this repository into your $DENSEPOSE, and replace corresponding paths specified in this file with yours:
diff --git a/CMakeLists.txt b/CMakeLists.txt
index 488ea86..b59d9bb 100644
--- a/CMakeLists.txt
+++ b/CMakeLists.txt
@@ -1,11 +1,24 @@
cmake_minimum_required(VERSION 2.8.12 FATAL_ERROR)
+# set caffe2 cmake path manually
+set(Caffe2_DIR "/path/to/anaconda3/envs/env_name/lib/pythonx.x/site-packages/torch/share/cmake/Caffe2")
+# set cuDNN path
+set(CUDNN_INCLUDE_DIR "/path/to/your/cudnn/include")
+set(CUDNN_LIBRARY "/path/to/your/libcudnn/libcudnn.so")
+include_directories("/path/to/anaconda3/envs/env_name/include")
+# add static protobuf library
+add_library(libprotobuf STATIC IMPORTED)
+set(PROTOBUF_LIB "/path/to/anaconda3/envs/env_name/lib/libprotobuf.a")
+set_property(TARGET libprotobuf PROPERTY IMPORTED_LOCATION "${PROTOBUF_LIB}")
+
# Find the Caffe2 package.
# Caffe2 exports the required targets, so find_package should work for
# the standard Caffe2 installation. If you encounter problems with finding
# the Caffe2 package, make sure you have run `make install` when installing
# Caffe2 (`make install` populates your share/cmake/Caffe2).
find_package(Caffe2 REQUIRED)
+include_directories(${CAFFE2_INCLUDE_DIRS})
if (${CAFFE2_VERSION} VERSION_LESS 0.8.2)
# Pre-0.8.2 caffe2 does not have proper interface libraries set up, so we
@@ -34,19 +47,19 @@ add_library(
caffe2_detectron_custom_ops SHARED
${CUSTOM_OPS_CPU_SRCS})
-target_link_libraries(caffe2_detectron_custom_ops caffe2_library)
+target_link_libraries(caffe2_detectron_custom_ops caffe2_library libprotobuf)
install(TARGETS caffe2_detectron_custom_ops DESTINATION lib)
# Install custom GPU ops lib, if gpu is present.
if (CAFFE2_USE_CUDA OR CAFFE2_FOUND_CUDA)
# Additional -I prefix is required for CMake versions before commit (< 3.7):
# https://github.com/Kitware/CMake/commit/7ded655f7ba82ea72a82d0555449f2df5ef38594
- list(APPEND CUDA_INCLUDE_DIRS -I${CAFFE2_INCLUDE_DIRS})
+ # list(APPEND CUDA_INCLUDE_DIRS -I${CAFFE2_INCLUDE_DIRS})
CUDA_ADD_LIBRARY(
caffe2_detectron_custom_ops_gpu SHARED
${CUSTOM_OPS_CPU_SRCS}
${CUSTOM_OPS_GPU_SRCS})
- target_link_libraries(caffe2_detectron_custom_ops_gpu caffe2_gpu_library)
+ target_link_libraries(caffe2_detectron_custom_ops_gpu caffe2_gpu_library libprotobuf)
install(TARGETS caffe2_detectron_custom_ops_gpu DESTINATION lib)
endif()Download the source code of PyTorch, then copy the following folders into $CAFFE2_INCLUDE_PATH/utils/ (referred to this issue and this issue which may be related to recent updates of PyTorch):
$ cp -r $PYTORCH/caffe2/utils/threadpool $CAFFE2_INCLUDE_PATH/utils/
$ cp -r $PYTORCH/caffe2/utils/math $CAFFE2_INCLUDE_PATH/utils/Compile the custom operator:
$ cd $DENSEPOSE/build
$ cmake .. && makeCheck that the custom operator tests pass:
$ python $DENSEPOSE/detectron/tests/test_zero_even_op.pyIf you installed the custom operator successfully without any error, you will see the following results in your console:
$ python detectron/tests/test_zero_even_op.py
[E init_intrinsics_check.cc:43] CPU feature avx is present on your machine, but the Caffe2 binary is not compiled with it. It means you may not get the full speed of your CPU.
[E init_intrinsics_check.cc:43] CPU feature avx2 is present on your machine, but the Caffe2 binary is not compiled with it. It means you may not get the full speed of your CPU.
[E init_intrinsics_check.cc:43] CPU feature fma is present on your machine, but the Caffe2 binary is not compiled with it. It means you may not get the full speed of your CPU.
............
----------------------------------------------------------------------
Ran 12 tests in 3.155s
OKPlease read Fetch DensePose data in the official installation documentation.
Please read Setting-up the COCO dataset in the official installation documentation.
Congratulations! Now you've installed DensePose successfully! :tada:
Happy Hunting DensePose!