NVlabs / eg3d

Other
3.17k stars 352 forks source link

Hope that this will be setup on Colab #1

Open GeorvityLabs opened 2 years ago

GeorvityLabs commented 2 years ago

Would be awesome if the Inversion Demo could be set up on colab. That way we could also try making 3d avatars of ourselfs with just a single input image.

natowi commented 2 years ago

Would be awesome if the Inversion Demo could be set up on colab.

Yes, like the one for https://github.com/facebookresearch/pifuhd

Looking forward to see the code for eg3d.

robbsaber commented 2 years ago

Any update?

cantonioupao commented 2 years ago

I think they won't really do this. Please refer to #21

osushilover commented 1 year ago

Is there anyone can make this code an colab notebook?

neverix commented 1 year ago

I made one, to be released in 3 days

GeorvityLabs commented 1 year ago

I made one, to be released in 3 days

looking forward to it!

neverix commented 1 year ago

Beta, let me know if anything breaks! https://colab.research.google.com/drive/1nu4maHdENDav54UUYk7H0MjC3IBP81yZ

osushilover commented 1 year ago

Beta, let me know if anything breaks! https://colab.research.google.com/drive/1nu4maHdENDav54UUYk7H0MjC3IBP81yZ

Thank you for making the Colab notebook! How can I use this notebook to generate novel views images and head around views videos like the one in #28 ? I would like to generate them from a single face photo of a particular person.

neverix commented 1 year ago

Beta, let me know if anything breaks! https://colab.research.google.com/drive/1nu4maHdENDav54UUYk7H0MjC3IBP81yZ

Thank you for making the Colab notebook! How can I use this notebook to generate novel views images and head around views videos like the one in #28 ? I would like to generate them from a single face photo of a particular person.

You should put that photo into this cell

image

(just remove the default URL link)

Then tweak focal length and make sure the optimization doesn't break. After that you can export the model and open it in something like Blender. I will probably add normalization and look around views like in #28 in the next version, thanks for the idea!

osushilover commented 1 year ago

Beta, let me know if anything breaks! https://colab.research.google.com/drive/1nu4maHdENDav54UUYk7H0MjC3IBP81yZ

Thank you for making the Colab notebook! How can I use this notebook to generate novel views images and head around views videos like the one in #28 ? I would like to generate them from a single face photo of a particular person.

You should put that photo into this cell

image

(just remove the default URL link)

Then tweak focal length and make sure the optimization doesn't break. After that you can export the model and open it in something like Blender. I will probably add normalization and look around views like in #28 in the next version, thanks for the idea!

Thank you for your reply. I see. I'd like to try the around view version. Once that's done, I want to be first in line to run it!

neverix commented 1 year ago

Thank you for your reply. I see. I'd like to try the around view version. Once that's done, I want to be first in line to run it!

Lookarounds are done #47

osushilover commented 1 year ago

Lookarounds are done #47

I've done, too! But The generated images do not resemble the person's face, have different faces mixed in, or have a lot of noise. exam01 https://streamable.com/qw5l4z And I would like to download "video.mp4" instead of "video.gif" if possible, cause I couldn't have found that mp4. What should I do?

neverix commented 1 year ago

It looks like your optimization broke (look at the upper left corner, it should be similar to upper right). You should tune parameters like focal_length and the optimization weights.

You can change the video save path in the cell below from video.gif to video.mp4 but I haven't tested it

osushilover commented 1 year ago

That's do a helpful advise! I'll study Stylegan2 more deeply.

luminohope commented 1 year ago

FYI We added additional scripts that can preprocess in-the-wild images compatible with the FFHQ checkpoints. https://github.com/NVlabs/eg3d/issues/18#issuecomment-1200366872

juanjomon commented 1 year ago

Thanks Neverix for your work. It is really appreciated. I use your Colab notebook and I feel like the images generated are always frontal view and the colours are usually dull. Why is this? . Also the inversion is not very accurate in terms of likeness. Is this due to the database? , Is there any way to obtain the same results as in the videos of the paper (different positions, full colour) and the same quality in inversion as in Stylegan3?. It would be great if this could be setup On the other hand, showing the depth maps is a great feature!. Thanks again for your work.

metekutlu07 commented 1 year ago

Thank you very much @neverix for the notebook. Is it possible to use another dataset, for example of flowers ? We are trying to convert images of flowers into basic 3D forms for an artistic project. It seems possible in their explanation but maybe not on the colab notebook ? I am pretty new on the subject. All help is greatly appreciated !

jasuriy commented 2 months ago

@neverix hi were you able to run the code before converting it into colab ? I was going to train it with my custom dataset so curious about training it?

neverix commented 2 months ago

No, I haven't trained my own EG3D model or used the repo outside of Colab.