Stuck when trying run_on_animals.py demo.

[vhvkhoa]

Hi, I am trying to run the demo code but I got this output and it stuck there like forever:

Iteration 3
shape_w 6000.0
9.49e+07 | cam_t_pos_0: 0.00e+00 | feq_0: 9.47e+02 | fpos_0: 0.00e+00 | kp_proj_0: 5.59e+07 | pose_limit_0: 4.72e+05 | pose_prior_0: 3.52e+07 | pose_prior_tail_0: 3.52e+05 | shape_prior_0: 1.13e+06 | trans_init_0: 1.80e+06
9.48e+07 | cam_t_pos_0: 0.00e+00 | feq_0: 9.44e+02 | fpos_0: 0.00e+00 | kp_proj_0: 5.48e+07 | pose_limit_0: 4.54e+05 | pose_prior_0: 3.58e+07 | pose_prior_tail_0: 3.58e+05 | shape_prior_0: 1.58e+06 | trans_init_0: 1.80e+06
beta here: -0.23, -0.43, -0.01, -0.54, -0.33, 0.14, 0.01, -0.17, 0.03, -0.07, -0.15, 0.08, 0.10, -0.00, -0.28, 0.14, -0.01, 0.20, -0.07, -0.03,
Scale 6
weights: s2m 2000.00 m2s 10000.00
nan | cam_t_pos_0: 0.00e+00 | feq_0: 9.44e+02 | fpos_0: 0.00e+00 | kp_proj_0: 5.48e+07 | m2s_0: nan | pose_limit_0: 4.54e+05 | pose_prior_0: 3.58e+07 | pose_prior_tail_0: 3.58e+05 | s2m_0: 1.88e+06 | shape_prior_0: 1.58e+06 | trans_init_0: 1.80e+06

Would you please give me a hint on what I am struggling of and how to get through it ? I think I have followed all the instructions given in README.md.

[silviazuffi]

Hi, Is this during the SMAL fitting or the refinement? The nan comes from the term m2s. You should add a breakpoint where the nan is generated, there is a division by the area of the model, it could be that in your case this value is zero. I suggest adding a small value to the denominator ad see if this fixes the problem.

On 18. Oct 2020, at 09:55, vhvkhoa notifications@github.com wrote:

Hi, I am trying to run the demo code but I got this output and it stuck there like forever:

nan | cam_t_pos_0: 0.00e+00 | feq_0: 9.44e+02 | fpos_0: 0.00e+00 | kp_proj_0: 5.48e+07 | m2s_0: nan | pose_limit_0: 4.54e+05 | pose_prior_0: 3.58e+07 | pose_prior_tail_0: 3.58e+05 | s2m_0: 1.88e+06 | shape_prior_0: 1.58e+06 | trans_init_0: 1.80e+06 Would you please give me a hint on what I am struggling of and how to get through it ? I think I have followed all the instructions given in README.md.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/silviazuffi/smalr_online/issues/2, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABNYVFJU4EDXTY7VWRFHY6LSLKNPBANCNFSM4SU5G76Q.

[vhvkhoa]

Hi, Is this during the SMAL fitting or the refinement? The nan comes from the term m2s. You should add a breakpoint where the nan is generated, there is a division by the area of the model, it could be that in your case this value is zero. I suggest adding a small value to the denominator ad see if this fixes the problem. …

Thank you for replying me. I have fixed the problem by removing the denominator if it's zero. May I ask you one more question, I want to visualize the 3D result mesh but I don't know how to do it and cannot find information in README.md. Could you give me some hint on how to visualize ?

[silviazuffi]

You can look at the function show_results.py

Best

On 19. Oct 2020, at 06:42, vhvkhoa notifications@github.com wrote:

Hi, Is this during the SMAL fitting or the refinement? The nan comes from the term m2s. You should add a breakpoint where the nan is generated, there is a division by the area of the model, it could be that in your case this value is zero. I suggest adding a small value to the denominator ad see if this fixes the problem. … <x-msg://258/#> Thank you for replying me. I have fixed the problem by removing the denominator if it's zero. May I ask you one more question, I want to visualize the 3D result mesh but I don't know how to do it and cannot find information in README.md. Could you give me some hint on how to visualize ?

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/silviazuffi/smalr_online/issues/2#issuecomment-711549167, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABNYVFMFV5RNZZTIF5RNXP3SLO7TFANCNFSM4SU5G76Q.

[xcyan]

@silviazuffi Btw, I have the same issue when running the demo script.

This happens when the rends[0] is a chumpy tensor with zeros everywhere.

See this function: https://github.com/silviazuffi/smalr_online/blob/master/src/smalr/silhouette_multi_model.py#L216-L227

[Autumnfall-Me]

@silviazuffi I have the same issue with vhvkhoa. Then I added a small value to the denominator, it seems to have been solved. But now I have a new issue, similar to the previous one. removing shoulders, often bad annotated Scale 6 Downweighting regularizers 0.9 weights: s2m 2.00 m2s 8.00 free_variables are dv nan | arap: 1.30e+00 | kp_proj_0: 6.66e+00 | kp_proj_1: 3.85e+00 | kp_proj_2: 4.42e+00 | lap: 7.29e+00 | m2s_0: nan | m2s_1: nan | m2s_2: nan | s2m_0: 1.27e+01 | s2m_1: 2.55e+00 | s2m_2: 5.64e+00 | sym_0: 1.95e+02 | sym_1: 1.01e+03 | sym_2: 1.80e+02

[silviazuffi]

Hi, In general when you have a nan there could be a denominator that goes to zero. Your problem is in the m2s function, which is computed as: GMOf(rends[i][:, :, 0] * dist_tsf[i], sigma[i]) / np.sqrt(n_model[i])

So you should check the value of n_model[I]. Or check if there are nan in rends[I].

About the question on pigs. A new shape is captured using only the silhouette, therefore you need to use images where the silhouette provides enough information to modify the original SMAL shape for animals that are not in the training set. I mean, if you want to generate the typical pig nose shape, you need to see the pig head from a side. I have added a pig example. Please remember that all the files and method can only be used for non commercial use.

On 29. Dec 2020, at 02:57, Autumnfall-Me notifications@github.com wrote:

hoa. Then I added a small value to the denominator, it seems to have been solved.

[Autumnfall-Me]

@silviazuffi, Thank you for your quick reply. I'll try it right away. Please don't worry, I promise not to use it for commercial purposes.

[silviazuffi]

Hi, If you need to add the small value in the denominator, it means that the renderer is not generating any silhouette. This could depend on the fact that the parameters used in the code (i.e. initial focal length) are not appropriate for your images. Or could depend from the viewing frustum. In all our experiments we never has this issue.

Best,

Silvia

On 11. Apr 2021, at 03:13, iegorval @.***> wrote:

@vhvkhoa https://github.com/vhvkhoa @xcyan https://github.com/xcyan @Autumnfall-Me https://github.com/Autumnfall-Me Hello, were you able to fix this issue? I had the same problem, and I have added the small value to the denominator in obj_m2s, so nan values disappeared, but the code still crashes when I try it on my images (worked fine on the examples though).

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/silviazuffi/smalr_online/issues/2#issuecomment-817230208, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABNYVFLE2WHTJRFXRH7DG53TIDZT5ANCNFSM4SU5G76Q.

[xiruiYan]

@silviazuffi

hi, when i run this code directly on your provided data, i met the same problem : nan | cam_t_pos_0: 0.00e+00 | feq_0: 9.44e+02 | fpos_0: 0.00e+00 | kp_proj_0: 5.48e+07 | m2s_0: nan | pose_limit_0: 4.54e+05 | pose_prior_0: 3.58e+07 | pose_prior_tail_0: 3.58e+05 | s2m_0: 1.88e+06 | shape_prior_0: 1.58e+06 | trans_init_0: 1.80e+06

and if i add the small value to the denominator, there is a new issue: free_variables are dv nan | arap: 1.47e-01 | kp_proj_0: 6.43e-02 | lap: 8.10e-01 | m2s_0: nan | s2m_0: 2.32e-01 | sym_0: 2.17e+01 | sym_1: 1.12e+02 | sym_2: 2.00e+01

can you help check what's the problem? thanks

[xiruiYan]

@silviazuffi, Thank you for your quick reply. I'll try it right away. Please don't worry, I promise not to use it for commercial purposes.

excuse me, i met the same problem with you, could you tell me how to solve this? thank you very much!

[silviazuffi]

Hi,

If you have that problem (a zero denominator) it means that the renderer is not working. You should always have a non-zero denominator, as the model should be always visible after initialisation. You can try placing the OpenDR import before all the other imports that use OpenGL (like the mesh viewer)

On 20. May 2021, at 10:04, xiruiYan @.***> wrote:

@silviazuffi https://github.com/silviazuffi, Thank you for your quick reply. I'll try it right away. Please don't worry, I promise not to use it for commercial purposes.

excuse me, i met the same problem with you, could you tell me how to solve this? thank you very much!

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/silviazuffi/smalr_online/issues/2#issuecomment-844837684, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABNYVFOGJTWZJ6IQQNWMNM3TOS7CBANCNFSM4SU5G76Q.

[xiruiYan]

@silviazuffi i checked all the inputs of coloredRenderer are not zero, while the output 'rends' are all zero. I try what you say but it doesn't work, maybe the use of colorrender is wrong or the version of OpenDR is wrong? rends = [ColoredRenderer( vc=np.ones_like(sv[i].r), v=sv[i], f=sv[i].f, camera=cameras[i], frustum=frustums[i], bgcolor=ch.array([0, 0, 0])) for i in range(nCameras)]

[xiruiYan]

@silviazuffi I find that all the rendering result img of OpenDr in the code are zero. i.e. all the outputs of render_mesh in myrender.py are zero... could you pls help check the problem? thanks!!

[silviazuffi]

Hi,

Your problem seems to be with OpenDR. I cannot reproduce your problem, but I suggest to first check if OpenDR works by running their demo (https://github.com/mattloper/opendr/blob/master/README.txt). Somebody pointed me to this repository: https://github.com/polmorenoc/opendr I have not checked it out, but maybe it can help.

Best

On 21. May 2021, at 05:30, xiruiYan @.***> wrote:

@silviazuffi https://github.com/silviazuffi I find that all the rendering result img of OpenDr in the code are zero. i.e. all the outputs of render_mesh in myrender.py are zero... could you pls help check the problem? thanks!!

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/silviazuffi/smalr_online/issues/2#issuecomment-845626705, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABNYVFP5GOA6LJJ6WC36SNTTOXHV7ANCNFSM4SU5G76Q.

[xiruiYan]

Hi, Your problem seems to be with OpenDR. I cannot reproduce your problem, but I suggest to first check if OpenDR works by running their demo (https://github.com/mattloper/opendr/blob/master/README.txt). Somebody pointed me to this repository: https://github.com/polmorenoc/opendr I have not checked it out, but maybe it can help. Best … On 21. May 2021, at 05:30, xiruiYan @.***> wrote: @silviazuffi https://github.com/silviazuffi I find that all the rendering result img of OpenDr in the code are zero. i.e. all the outputs of render_mesh in myrender.py are zero... could you pls help check the problem? thanks!! — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#2 (comment)>, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABNYVFP5GOA6LJJ6WC36SNTTOXHV7ANCNFSM4SU5G76Q.

thanks for your help! I solve this problem by add these imports at the begin of run_on_animals.py

from opendr.renderer import ColoredRenderer from opendr.renderer import TexturedRenderer from opendr.lighting import LambertianPointLight

[daisyranc]

@silviazuffi I have the same issue with vhvkhoa. Then I added a small value to the denominator, it seems to have been solved. But now I have a new issue, similar to the previous one. removing shoulders, often bad annotated Scale 6 Downweighting regularizers 0.9 weights: s2m 2.00 m2s 8.00 free_variables are dv nan | arap: 1.30e+00 | kp_proj_0: 6.66e+00 | kp_proj_1: 3.85e+00 | kp_proj_2: 4.42e+00 | lap: 7.29e+00 | m2s_0: nan | m2s_1: nan | m2s_2: nan | s2m_0: 1.27e+01 | s2m_1: 2.55e+00 | s2m_2: 5.64e+00 | sym_0: 1.95e+02 | sym_1: 1.01e+03 | sym_2: 1.80e+02 @Autumnfall-Me

Have you solved the problem? I have met the same problem as you. And I cannot solve it even if I set a small value on render[I] when it == 0.