search

gafniguy / 4D-Facial-Avatars

Dynamic Neural Radiance Fields for Monocular 4D Facial Avater Reconstruction
670 stars 67 forks source link

Question about Test speed #55

Closed vito929 closed 1 year ago

vito929 commented 1 year ago

Hi, After obtaining a model trained 400k epoches, I run eval_transformed_rays.py . It seems that time to render each frame is about 15 seconds (I use GeForce RTX 3060 to test), which means to render a 20-second video with 1000 frames I need to spend a couple of hours. Is it normal? May I ask how long does it take for you to test per frame? Could you give some hints about how to accelerate the test phase and is it possible to use it in real time in practice (now or in the future)? Thanks!

gafniguy commented 1 year ago

Hi, thanks for your questions.

This of course depends on the resolution you're rendering at, but the numbers sound right. Was taking me 15-30 seconds per frame. This is a NeRF thing in general, not specific to this project.

There are accelerated Nerf variants for real time rendering, some research works combine them with dynamic avatars. But these weren't around when this paper was worked on, so there's only standard nerf here and not a fast one.

You might want to take a look at e.g. Insta (zielonka et al) and others they cite.

Anyhow, if you have access to multiple GPUs you can modify the eval script to chunk it up and get the video out quicker.

Regards, Guy

Am 22.02.2023 10:33 schrieb vito929 @.***>:

Hi, After obtaining a model trained 400k epoches, I run eval_transformed_rays.py . It seems that time to render each frame is about 15 seconds (I use GeForce RTX 3060 to test), which means to render a 20-second video with 1000 frames I need to spend a couple of hours. Is it normal? May I ask how long does it take for you to test per frame? Could you give some hints about how to accelerate the test phase and is it possible to use it in real time in practice (now or in the future)? Thanks!

— Reply to this email directly, view it on GitHubhttps://github.com/gafniguy/4D-Facial-Avatars/issues/55, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AI7AQ5EDOGMW5477HUNRT63WYXMP5ANCNFSM6AAAAAAVECYAHU. You are receiving this because you are subscribed to this thread.Message ID: @.***>

vito929 commented 1 year ago

Thank you for replaying. I also read your paper and got a question. In Fig.2, it says that shape of Intrinsics is [3,3] but the from the the code I think the so called Intrinsics is variable hw[2] with the shape of [4,] (intrinsics = [fx fy cx cy] where cx cy in [0,1] relative to image size). So could you help explain why and wether my understanding is wrong? Thanks!

gafniguy commented 1 year ago

it's a sparser way to save it, is it is of the form

image