Whether the position maps contains pose variation and the demonstration problem.

[ChunLLee]

Thanks for sharing your marvellous work! I have encountered a problem, could anybody help? I tried to train the model and demonstrate the vertices in the same way as in demo.py. However, the visual results is not as expected. To be specific, the output position map of the network does not contain poses, therefore in the showing process, it cannot align with the faces in the original image. The vertices are all frontal. While the demonstration results of the demo.py contains poses, and can perfectly align with the input faces.

In the demo.py, the input of the network are processed with:

# the core: regress position map if args.isDlib: max_size = max(image.shape[0], image.shape[1]) if max_size> 1000: image = rescale(image, 1000./max_size) image = (image*255).astype(np.uint8) pos = prn.process(image) # use dlib to detect face else: if image.shape[0] == image.shape[1]: image = resize(image, (256,256)) pos = prn.net_forward(image/255.) # input image has been cropped to 256x256 else: box = np.array([0, image.shape[1]-1, 0, image.shape[0]-1]) # cropped with bounding box pos = prn.process(image, box)

where 'pos' is the output position map

and then 'pos' is processed with vertices = prn.get_vertices(pos), where: `def get_vertices(self, pos): ''' Args: pos: the 3D position map. shape = (256, 256, 3). Returns: vertices: the vertices(point cloud). shape = (num of points, 3). n is about 40K here. ''' all_vertices = np.reshape(pos, [self.resolution_op**2, -1]) vertices = all_vertices[self.face_ind, :]

    return vertices`

Finally, the vertices are circled in the original image, as: cv2.imshow('dense alignment', plot_vertices(image, vertices))

In the whole process, I did not find anything about pose transformation, so I am wondering, whether the position map of the training data contains pose or not, or did I miss anything concerning pose transformation in the demonstration process?

[ChunLLee]

@YadiraF @jnulzl

[ChunLLee]

Solved. The position maps of the training data are transformed from projected vertices, which are already aligned with the input image.