search

EnVision-Research / Lotus

Official Implementation of LOTUS: Diffusion-based Visual Foundation Model for High-quality Dense Prediction
https://lotus3d.github.io
Apache License 2.0
398 stars 22 forks source link

Is it possible to get more shots from various angles ? #19

Open altava-sgp opened 3 days ago

altava-sgp commented 3 days ago

At first thanks for your great job! 👍

I have a question. Is it possible to get more shots from various angles ? A only single shot is possible ?

Please excuse me if I am missing anything.

haodong2000 commented 18 hours ago

Hi @altava-sgp thanks so much for your interest! Sorry for my late response.

Do you mean to get multi views of depth and normal given a single RGB image?

For depth estimation: you can lift the depth maps into 3D point clouds using camera intrinsics and then render new depth maps or RGB from different views. As shown in this issue.

For normal, if you want mulri views of normal maps, you probably should do: normal -> depth -> 3D point clouds -> depth (new view) -> normal (new view). You can refer this repo for normal2depth and this repo for depth2normal.

Best,

altava-sgp commented 8 hours ago

@haodong2000 Thanks for your answer. I will try as you said. 👍

altava-sgp commented 8 hours ago

@haodong2000 I also hope Lotus can generate various normals like I said. It will be much helpful for AI generation. Please consider this seriously. 🙏

altava-sgp commented 8 hours ago

Like these. normals_back_masked normals_front_left_masked normals_front_masked normals_front_right_masked normals_left_masked normals_right_masked