search

ZHU-Zhiyu / NVS_Solver

Source code of paper "NVS-Solver: Video Diffusion Model as Zero-Shot Novel View Synthesizer"
186 stars 1 forks source link

What is "NVS-Solver for abitrary trajectory"? #12

Closed tsuJohc closed 6 days ago

tsuJohc commented 1 week ago

Hi, Thanks for releasing such good work!

I noticed there is a term "Release NVS-Solver for abitrary trajectory" in the README's "To do list".

I wonder what is the differences between the "NVS-Solver for abitrary trajectory" and released codes (single image, multiple images).

Because I think we can implement the abitrary trajectory by modifying poses in the "single-image" script, or giving two abitrary poses and images in "multi-images" script.

Or is there any special optimization strategy for the abitrary-trajectory NVS

mengyou2 commented 6 days ago

Hi, thanks for your interest in our project.

In the original code for single images and dynamic images, novel poses were generated following an ellipse pattern, with the config only requiring the radius of the ellipse.

The recently uploaded arbitrary trajectory code, svd_interpolate_single_img_traj, changes the pose generation approach. By inputting the trajectory end position, you can now generate the corresponding video without following the ellipse pattern. We didn't implement a special optimization strategy; this change was made for convenient inference. As described in "Our pipeline for arbitrary trajectory," you can generate novel views of the given image from any camera position.

tsuJohc commented 6 days ago

Thanks for the response!