TimoBolkart / voca

This codebase demonstrates how to synthesize realistic 3D character animations given an arbitrary speech signal and a static character mesh.
https://voca.is.tue.mpg.de/en
1.15k stars 273 forks source link

a Gaussian filtering (across the sequence) to mitigate capture noise #37

Closed ZerRui closed 4 years ago

ZerRui commented 4 years ago

to get the Unposed Cleaned Data, we fix for each sequence (within the Unposed Data) the neck boundary vertices and the ear vertices to a fixed locatio, and apply to the region around the eyes a Gaussian filtering (across the sequence) to mitigate capture noise.

In Issue #28 Can you reveal the code for this step of Gaussian filtering? Thank you

TimoBolkart commented 4 years ago

We just load a sequence of vertices and run for each vertex for each coordinate (i.e. across the whole sequence) the gaussian_filter function from scipy.ndimage.filters The code is something like this for sequence_verts be the num_vertices x 3 x num_frames tensor

    new_verts = np.zeros_like(sequence_verts)
    for i in range(num_verts):
         for j in range(verts_dim):
             point_smooth = sequence_verts[i, j, :]
             for iter in range(num_iter):
                 point_smooth = gaussian_filter(point_smooth, sigma=sigma)
                 new_verts[i, j, :] = point_smooth

Does this answer your question?