How to generate normalized 2D template from 3D template

[qaz8788817]

Hi. I hope this message finds you well. I apologize for reaching out again. I'm writing regarding Section 3.1 of the paper, where it mentions the process of converting a 3D template into a 2D template. Could you kindly provide me with more detailed steps or perhaps some sample code?

I understand that my dataset differs from the FaceScape dataset mentioned in the repository, so the templates provided there might not be suitable for my use.

I apologize for the repeated inquiries. If it's inconvenient to discuss this publicly, feel free to email me at gn00088507@gmail.com.

Thank you very much for your time and assistance.

[NaughtyZZ]

I suggest that you would be better to rescale the provided template and make it suitable for your data. The "unfolding'' of a 3D template to obtain a 2D one requires quite a lot manual efforts. It also requires that the 3D template structure is nearly symmetric.

[NaughtyZZ]

I can send you some codes via email. It may be useful to you if your template is other type of geometric data rather than human face.

[qaz8788817]

I suggest that you would be better to rescale the provided template and make it suitable for your data. The "unfolding'' of a 3D template to obtain a 2D one requires quite a lot manual efforts. It also requires that the 3D template structure is nearly symmetric.

Thank you for your suggestions and patience.

[qaz8788817]

I can send you some codes via email. It may be useful to you if your template is other type of geometric data rather than human face.

Hi. I'm writing to inquire about the email you mentioned sending me a few days ago. I've checked my inbox, but it seems I haven't received it yet. Could you please confirm if you've already sent it or if there were any issues on your end?

My email is: gn00088507@gmail.com

Thank you for your assistance!

[NaughtyZZ]

I send you via email several times but it seems to be blocked by the gmail server.

[qaz8788817]

I send you via email several times but it seems to be blocked by the gmail server.

If you send the mail to here: b0929022@cgu.edu.tw

Thank you.

[qaz8788817]

Hi, @NaughtyZZ . Hope all is well. I encountered some issues while executing the zip file you provided, and I've sent an email to your inbox (fanzhenfeng@ict.ac.cn) regarding the difficulties I faced. Here is my email: b0929022@cgu.edu.tw

I would appreciate it if you could take a moment to check it out and provide some advice. Thank you for your assistance.

[NaughtyZZ]

The answers are as follows:

1. "Gfield.mat" is a precomputed data whose elements are composed of the vertex-to-vertex geodesic distance.

2. While some codes are fragmented and some involves manual operations (e.g. rearranging the key vertex and edge locations), I am sorry that I cannot provide more details. The core pipeline is included in Section 3.1 of the paper and almost all the details are included in this package I sent to you.

3. "writeMDSPly.m" should be the first as a simple harmonic parametrization method. "normalized_2D_final.ply" is generated in the excute or''excute2'' sub-folder after the rearrangement of some key vertices.

4. 'computeVertexRing_k' is to compute the 1-ring neighbor of each vertex.

I hope that these answers are helpful to you.

[qaz8788817]

Hi, @NaughtyZZ thank you very much for your response last time; I found it very inspiring. This time, I would like to ask about the parameters in idxE.mat. Do they represent vertex indices for different regions? This is my assumption: idxE contains vertex indices for all boundaries, but I'm not entirely sure if E1, E2, E3, and E4 correspond to the right eye, outer, mouth, and left eye respectively. And could you please clarify how to define the boundaries on the normalized_2D.ply file?

Thank you.

[NaughtyZZ]

E1, E2, E3, and E4 correspond to the left eye, outer, mouth, and right eye, respectively. I suggest that you may change the color of the vertices and re-write the *.ply file to verify it. The boundary includes the edge vetices actually. The edges of these vertices are not shared by 2 triangles.

[qaz8788817]

E1, E2, E3, and E4 correspond to the left eye, outer, mouth, and right eye, respectively. I suggest that you may change the color of the vertices and re-write the *.ply file to verify it. The boundary includes the edge vetices actually. The edges of these vertices are not shared by 2 triangles.

Then, how do you classify E1, E2, E3, and E4? This is my main point of confusion at the moment. Additionally, when I used my method to generate idxE, I found that the number of boundary vertex indices I collected differs by 23 from the number in your idxE. Could this be due to different algorithms, or have I overlooked some details in my definition? Here is the main function: ``% initialize count adjacent_triangles_count = zeros(size(shape', 1), 1);

% count each vertex index in each face for i = 1:size(triList') % get the current triangle's 3 index triangle_indices = triList(:, i); % update the counting adjacent_triangles_count(triangle_indices) = adjacent_triangles_count(triangle_indices) + 1; end

% find the edge vertex idxE = find(adjacent_triangles_count < 4);``

Thank you. If I've caused any inconvenience, I sincerely apologize, and I appreciate your willingness to help me with my questions.

[NaughtyZZ]

(1)The original and most reliable way to verify this is to show the resulted edge vertices. I first find the edges that are not shared by two triangles (be careful about the order of 2 vetices on a edge). Then I got the boundary vertices from these edges. In your codes, you direct deduce the edge vetices from triangles， the problem maybe that a vertex shared by more than 3 triangles does not equals a edge vetex.

(2)After obtaining the boundary edges, I use the connectivity relationship to extract E1 to E4. Then I visually verify that they correspond to different regions (the left eye, outer, mouth, and right eye).

[qaz8788817]

@NaughtyZZ , thank you very much for your response. I've regenerated idxE using my own method, but due to my lack of understanding, I'm still unsure how to proceed with idxE1 to idxE4 in terms of computation.

I believe generating E1 to E4 may require the use of the computeVertexRing_k function. I attempted to validate my computeVertexRing_k using compute_vertex_weight.m and compute_local_planes.m, but I couldn't obtain a jetMap_w.ply file similar to the one in your attachment. Instead, it consistently displays a single color (red).

Could you please help me understand the reason behind this?

And I've been endeavoring to reconstruct the *.exe files within the excute and excute2 subfolders (because the exe cannot work on my computer), but unfortunately, my efforts have been met with some challenges. Despite my earnest attempts and even delving into crafting my code, I've been unable to achieve successful reconstruction. I'm keen to obtain access to the relevant subfunctions, as I believe it may aid in resolving this issue. Any assistance or guidance in this matter would be greatly appreciated.

Here is my computeVertexRing_k: `function ring = computeVertexRing_k(triList, k, vertices) % triList: List of triangles where each row contains the indices of three vertices of a triangle % k: Number of neighbor rings to compute % vertices: List of vertex indices used to compute neighbor rings

numVertices = length(vertices);
ring = zeros(numVertices, k);

% build neighbor ring
for i = 1:numVertices
    v = vertices(i);
    neighbors = find(sum(ismember(triList, v), 2)); % Find all triangles that contain the current vertex
    neighborVertices = unique(triList(neighbors, :)); % Get neighboring vertices
    % neighborVertices(neighborVertices == v) = []; % Remove the current vertex itself

    % Add neighbor vertices to the ring
    ring(i, 1:min(k, length(neighborVertices))) = neighborVertices(1:min(k, end));
end

end`

Your support has been invaluable.

[NaughtyZZ]

When you write the *ply file, you only require to change the color of the corresponding vertices, e.g. the common [128 128 128] (in the texture variable) to a different specified color. Then you can verify whether the vertices you find are correct.

[NaughtyZZ]

The corresponding vertices are indexed by idxE1, idxE2,...

[qaz8788817]

Hi @NaughtyZZ , I have generated facial models with different expression variations, but I encountered abnormal meshes in some areas. Could you please provide me with some suggestions on how to resolve these abnormal meshes? Here is the output of one of the expressions when opened using MeshLab: Currently, I suspect that the possible cause might be due to the positions of landmarks or the algorithm used in Gfield.mat, which could result in such training outcomes. Thanks.

[NaughtyZZ]

Which model do you use? The one I provided or that you trained yourself. I suggest that you might need to fine-tune the training process considering the later case, since it is a common routine for GAN.

[qaz8788817]

The model I'm using is self-generated, and both the 2D template and 3D template are regenerated by myself.