Invalid 3D mesh models for DensePose CSE

[sh8]

Instructions To Reproduce the 🐛 Bug:

Download a 3D mesh model of smpl_27554 from here. All vertex coordinates and faces are somehow filled with 0.

{'vertices': array([[ 0.,  0.,  0.],
       [ 0.,  0.,  0.],
       [ 0.,  0.,  0.],
       ...,
       [-0.,  0.,  0.],
       [-0.,  0.,  0.],
       [-0.,  0.,  0.]]), 'faces': array([[0, 0, 0],
       [0, 0, 0],
       [0, 0, 0],
       ...,
       [0, 0, 0],
       [0, 0, 0],
       [0, 0, 0]])}

Expected behavior:

The 3D mesh model should have correct vertex coordinates and faces.

Environment:

Provide your environment information using the following command:

----------------------  -----------------------------------------------------------------------------------------
sys.platform            linux
Python                  3.8.5 (default, Apr 20 2021, 20:29:03) [GCC 10.2.0]
numpy                   1.21.0
detectron2              0.4.1 @/home/detectron2/detectron2
Compiler                GCC 11.1
CUDA compiler           CUDA 10.1
detectron2 arch flags   7.0
DETECTRON2_ENV_MODULE   <not set>
PyTorch                 1.9.0+cu102 @/home/detectron2/.venv/lib/python3.8/site-packages/torch
PyTorch debug build     False
GPU available           True
GPU 0,1                 NVIDIA GeForce RTX 2080 SUPER (arch=7.5)
CUDA_HOME               /opt/cuda-10.1
Pillow                  8.2.0
torchvision             0.10.0+cu102 @/home/detectron2/.venv/lib/python3.8/site-packages/torchvision
torchvision arch flags  3.5, 5.0, 6.0, 7.0, 7.5
fvcore                  0.1.5.post20210624
iopath                  0.1.8
cv2                     4.5.2
----------------------  -----------------------------------------------------------------------------------------
PyTorch built with:
  - GCC 7.3
  - C++ Version: 201402
  - Intel(R) Math Kernel Library Version 2020.0.0 Product Build 20191122 for Intel(R) 64 architecture applications
  - Intel(R) MKL-DNN v2.1.2 (Git Hash 98be7e8afa711dc9b66c8ff3504129cb82013cdb)
  - OpenMP 201511 (a.k.a. OpenMP 4.5)
  - NNPACK is enabled
  - CPU capability usage: AVX2
  - CUDA Runtime 10.2
  - NVCC architecture flags: -gencode;arch=compute_37,code=sm_37;-gencode;arch=compute_50,code=sm_50;-gencode;arch=compute_60,code=sm_60;-gencode;arch=compute_70,code=sm_70
  - CuDNN 7.6.5
  - Magma 2.5.2
  - Build settings: BLAS_INFO=mkl, BUILD_TYPE=Release, CUDA_VERSION=10.2, CUDNN_VERSION=7.6.5, CXX_COMPILER=/opt/rh/devtoolset-7/root/usr/bin/c++, CXX_FLAGS= -Wno-deprecated -fvisibility-inlines-hidden -DUSE_PTHREADPOOL -fopenmp -DNDEBUG -DUSE_KINETO -DUSE_FBGEMM -DUSE_QNNPACK -DUSE_PYTORCH_QNNPACK -DUSE_XNNPACK -DSYMBOLICATE_MOBILE_DEBUG_HANDLE -O2 -fPIC -Wno-narrowing -Wall -Wextra -Werror=return-type -Wno-missing-field-initializers -Wno-type-limits -Wno-array-bounds -Wno-unknown-pragmas -Wno-sign-compare -Wno-unused-parameter -Wno-unused-variable -Wno-unused-function -Wno-unused-result -Wno-unused-local-typedefs -Wno-strict-overflow -Wno-strict-aliasing -Wno-error=deprecated-declarations -Wno-stringop-overflow -Wno-psabi -Wno-error=pedantic -Wno-error=redundant-decls -Wno-error=old-style-cast -fdiagnostics-color=always -faligned-new -Wno-unused-but-set-variable -Wno-maybe-uninitialized -fno-math-errno -fno-trapping-math -Werror=format -Wno-stringop-overflow, LAPACK_INFO=mkl, PERF_WITH_AVX=1, PERF_WITH_AVX2=1, PERF_WITH_AVX512=1, TORCH_VERSION=1.9.0, USE_CUDA=ON, USE_CUDNN=ON, USE_EXCEPTION_PTR=1, USE_GFLAGS=OFF, USE_GLOG=OFF, USE_MKL=ON, USE_MKLDNN=ON, USE_MPI=OFF, USE_NCCL=ON, USE_NNPACK=ON, USE_OPENMP=ON,

[sh8]

Looks like the same problem is happening to other mesh models of animals

[vkhalidov]

Hi @sh8, we cannot release vertex and face data for all the meshes due to license restrictions. We provide annotations and various data used within DensePose CSE pipeline (e.g. geodesic distance data, symmetry transforms, texture coordinates, Laplace-Beltrami operator features etc). Some meshes (like SMPL) are available for download externally.

[sh8]

Thank you for the quick reply! Could you let me know how did you convert the original SMPL mesh to the one used in this repo? It looks like the number of meshes is different between them, but there is no information on how to convert it.

[vkhalidov]

Indeed, for DensePose we use an upsampled mesh that contains 27554 vertices. The first 6890 vertices on the upsampled mesh correspond to the 6890 canonical SMPL vertices. The other ones can be mapped to the closest one of those 6890 vertices by using the geodesic distances tensor (available here, see data/meshes/builtin.py)

[sh8]

Thank you so much!