fit_textured_mesh.ipynb file cant import the TexturesVector class in google colab + some guidance is requested

[ShivamSRS]

This seem to require 2 labels, bugs and how to

🐛 Bugs / Unexpected behaviors

I ran the fit_textured_mesh notebook in tutorials of the git repo in google colaboratory, however after installing the necessary installations of cell 1 the code gave an error in the cell 2 where all the imports were done.

Instructions To Reproduce the Issue:

Run the tutorial In colab with gpu selected as hardware accelerator ( I got a tesla k80 ).

1. Any changes you made (git diff) or code you wrote None

2. The exact command(s) you ran:

cell 1:

!pip install torch torchvision !pip install 'git+https://github.com/facebookresearch/pytorch3d.git@stable'

cell 2:

import os import torch import matplotlib.pyplot as plt from skimage.io import imread

from pytorch3d.utils import ico_sphere import numpy as np from tqdm.notebook import tqdm

from pytorch3d.io import load_objs_as_meshes, save_obj

from pytorch3d.loss import ( chamfer_distance, mesh_edge_loss, mesh_laplacian_smoothing, mesh_normal_consistency, )

from pytorch3d.structures import Meshes from pytorch3d.renderer import ( look_at_view_transform, OpenGLPerspectiveCameras, PointLights, DirectionalLights, Materials, RasterizationSettings, MeshRenderer, MeshRasterizer,
SoftPhongShader, SoftSilhouetteShader, SoftPhongShader, TexturesVertex )

import sys import os sys.path.append(os.path.abspath(''))

3. What you observed (including the full logs):

output of cell 1 :

Requirement already satisfied: torch in /usr/local/lib/python3.6/dist-packages (1.6.0+cu101) Requirement already satisfied: torchvision in /usr/local/lib/python3.6/dist-packages (0.7.0+cu101) Requirement already satisfied: future in /usr/local/lib/python3.6/dist-packages (from torch) (0.16.0) Requirement already satisfied: numpy in /usr/local/lib/python3.6/dist-packages (from torch) (1.18.5) Requirement already satisfied: pillow>=4.1.1 in /usr/local/lib/python3.6/dist-packages (from torchvision) (7.0.0) Collecting git+https://github.com/facebookresearch/pytorch3d.git@stable Cloning https://github.com/facebookresearch/pytorch3d.git (to revision stable) to /tmp/pip-req-build-1y6sl2_t Running command git clone -q https://github.com/facebookresearch/pytorch3d.git /tmp/pip-req-build-1y6sl2_t Running command git checkout -q 686c8666d31d932ed42d3cd7319f249fc75e89a9 Requirement already satisfied: torchvision>=0.4 in /usr/local/lib/python3.6/dist-packages (from pytorch3d==0.2.0) (0.7.0+cu101) Collecting fvcore Downloading https://files.pythonhosted.org/packages/81/ee/3709513b414c2b525e4f03c45215579f93f41dba797f0e5fe539e6bc92b7/fvcore-0.1.1.post20200716.tar.gz Requirement already satisfied: pillow>=4.1.1 in /usr/local/lib/python3.6/dist-packages (from torchvision>=0.4->pytorch3d==0.2.0) (7.0.0) Requirement already satisfied: torch==1.6.0 in /usr/local/lib/python3.6/dist-packages (from torchvision>=0.4->pytorch3d==0.2.0) (1.6.0+cu101) Requirement already satisfied: numpy in /usr/local/lib/python3.6/dist-packages (from torchvision>=0.4->pytorch3d==0.2.0) (1.18.5) Collecting yacs>=0.1.6 Downloading https://files.pythonhosted.org/packages/38/4f/fe9a4d472aa867878ce3bb7efb16654c5d63672b86dc0e6e953a67018433/yacs-0.1.8-py3-none-any.whl Collecting pyyaml>=5.1 Downloading https://files.pythonhosted.org/packages/64/c2/b80047c7ac2478f9501676c988a5411ed5572f35d1beff9cae07d321512c/PyYAML-5.3.1.tar.gz (269kB) |████████████████████████████████| 276kB 5.6MB/s Requirement already satisfied: tqdm in /usr/local/lib/python3.6/dist-packages (from fvcore->pytorch3d==0.2.0) (4.41.1) Collecting portalocker Downloading https://files.pythonhosted.org/packages/89/a6/3814b7107e0788040870e8825eebf214d72166adf656ba7d4bf14759a06a/portalocker-2.0.0-py2.py3-none-any.whl Requirement already satisfied: termcolor>=1.1 in /usr/local/lib/python3.6/dist-packages (from fvcore->pytorch3d==0.2.0) (1.1.0) Requirement already satisfied: tabulate in /usr/local/lib/python3.6/dist-packages (from fvcore->pytorch3d==0.2.0) (0.8.7) Requirement already satisfied: future in /usr/local/lib/python3.6/dist-packages (from torch==1.6.0->torchvision>=0.4->pytorch3d==0.2.0) (0.16.0) Building wheels for collected packages: pytorch3d, fvcore, pyyaml Building wheel for pytorch3d (setup.py) ... done Created wheel for pytorch3d: filename=pytorch3d-0.2.0-cp36-cp36m-linux_x86_64.whl size=10290212 sha256=a6162443d622a9b22dfbf5987866b2f062cf2a676c07a3f48281999fb8ba8531 Stored in directory: /tmp/pip-ephem-wheel-cache-ukmzoh3c/wheels/89/69/08/d864f516508b5d943259c6088baa150ebdd3659b5dde4e3571 Building wheel for fvcore (setup.py) ... done Created wheel for fvcore: filename=fvcore-0.1.1.post20200716-cp36-none-any.whl size=42325 sha256=ee7eff2559c3b3fd15a240925d3c743e36718c098b9e4d9c2c54426ada7ec95c Stored in directory: /root/.cache/pip/wheels/81/99/f4/42a6bef61c07b3d78dfe6d7ebff259444c4526504cf72378d7 Building wheel for pyyaml (setup.py) ... done Created wheel for pyyaml: filename=PyYAML-5.3.1-cp36-cp36m-linux_x86_64.whl size=44621 sha256=2ba9b500c5678cc80f492d512c8f539fa983e05d3869dcf039755871d98e330e Stored in directory: /root/.cache/pip/wheels/a7/c1/ea/cf5bd31012e735dc1dfea3131a2d5eae7978b251083d6247bd Successfully built pytorch3d fvcore pyyaml Installing collected packages: pyyaml, yacs, portalocker, fvcore, pytorch3d Found existing installation: PyYAML 3.13 Uninstalling PyYAML-3.13: Successfully uninstalled PyYAML-3.13 Successfully installed fvcore-0.1.1.post20200716 portalocker-2.0.0 pytorch3d-0.2.0 pyyaml-5.3.1 yacs-0.1.8

output of cell 2 :

ImportError Traceback (most recent call last)

in () 20 # Data structures and functions for rendering 21 from pytorch3d.structures import Meshes ---> 22 from pytorch3d.renderer import ( 23 look_at_view_transform, 24 OpenGLPerspectiveCameras, ImportError: cannot import name 'TexturesVertex' I would be highly obliged if any one helps me I am actually pretty new with neural rendering and 3D deep learning and know little about it, I would highly appreciate if someone can guide me on getting started with it and making custom projects with pytorch 3D. My final year undergraduate project basically consists of transferring facial expressions from one person to another and i have seen many papers doing the same with neural rendering but most of them have been posted with incomplete implementations in their github repositories. What should be the roadmap if i have to do the same using pytorch 3D? I am under the impression it should be something along the lines of making a 3d mesh of a face somehow and then taking source's photo or video and mapping their face to the mesh while repeating the same for the target and then somehow encoding the difference in motion of each point on meshes and warping it on the target face mesh and then convert it into a video? Please help me with resources on how do i do this with pytorch 3D and how do i get the face meshes. I shall be highly grateful for any help whatsoever

[bottler]

I think you are using the latest version of the tutorial from the master branch of github, which is not what you want. If you go to pytorch3d.org and follow the link to open the tutorial in colab you will get the stable version, which will work with the stable version of pytorch3d. (You can also download the tutorial from https://github.com/facebookresearch/pytorch3d/tree/stable/docs/tutorials ).

[ShivamSRS]

Thanks @bottler for responding to me but there doesn't seem to be this particular tutorial (fit_textured_mesh.ipynb ) at the link you mentioned. Could please guide me how do I run, in the stable version as u said, that specific tutorial? The issue majorly seems to be an import error

[bottler]

Sorry, that particular tutorial has been written since the last release, and is not designed to work with the release.

You can install the latest version of pytorch3d in colab (instead of stable) by using

!pip install 'git+https://github.com/facebookresearch/pytorch3d.git'

instead of

!pip install 'git+https://github.com/facebookresearch/pytorch3d.git@stable'

[ShivamSRS]

Thanks a lot @bottler I was able to run it by using the latest repo using the command you mentioned, the ImportError for the TexturesVertex is resolved but there seems to be a problem with the constructor of the same class.

In the 3rd section of the notebook Mesh and texture prediction via textured rendering The forward pass for loop has the code line

new_src_mesh.textures = TexturesVertex(verts_rgb=sphere_verts_rgb)

where I am getting the following error:

TypeError Traceback (most recent call last)

in () 9 10 # Add per vertex colors to texture the mesh ---> 11 new_src_mesh.textures = TexturesVertex(verts_rgb=sphere_verts_rgb) 12 13 # Losses to smooth /regularize the mesh shape TypeError: __init__() got an unexpected keyword argument 'verts_rgb' The constructor doesn't seem to have this keyword in the arguments in textures.py file of the repo too. also the same issue then gives the following error in the line code : visualize_prediction(new_src_mesh, renderer=renderer_textured, silhouette=False) error : /usr/local/lib/python3.6/dist-packages/pytorch3d/structures/meshes.py in sample_textures(self, fragments) 1498 ) 1499 else: -> 1500 raise ValueError("Meshes does not have textures") 1501 1502 ValueError: Meshes does not have textures which I am assuming is a result of the new_src_mesh.textures not being initialized due to wrong arguments passed in the constructor. Thank you for being so prompt and patient while helping me. I really appreciate it.

[bottler]

I think you need to replace verts_rgb by verts_features. The second error is probably caused by bypassing the first.

[nikhilaravi]

@ShivamSRS were all the problems resolved? Can we close this issue?

[ShivamSRS]

Yes thank you for all the help @nikhilaravi @bottler . You all are great!