zju3dv / ENeRF

SIGGRAPH Asia 2022: Code for "Efficient Neural Radiance Fields for Interactive Free-viewpoint Video"
https://zju3dv.github.io/enerf
Other
413 stars 28 forks source link

The vedio of the ZJU-Mocap #27

Closed ds22058 closed 1 year ago

ds22058 commented 1 year ago

Hi, I try to prepare my own Mocap dataset to run your code but the youtube link in https://chingswy.github.io/easymocap-public-doc/quickstart/capture_youtube.html is private or not exist. Could you please release the multiview vedio of your Mocap vedio so that I can have a try on preparing Mocap dataset. Thanks in advance!

ds22058 commented 1 year ago

I have another question. Following the https://chingswy.github.io/easymocap-public-doc/quickstart/prepare_mocap.html I get imageHowever in Core_view313,

image

it has annots.npy, annots.json and files like new_vertices and so on. Without them I can't run the'gui_human.py'. How can I get them.

ds22058 commented 1 year ago

Also, I wanna know how to make my own enerf_outdoor dataset

haotongl commented 1 year ago

https://chingswy.github.io/easymocap-public-doc/quickstart/prepare_mocap.html

Hi, ENeRF use the dataset format of NeuralBody. Please use this script to convert a dataset of MoCap format to NeuralBody format. https://chingswy.github.io/easymocap-public-doc/quickstart/tools.html

haotongl commented 1 year ago

Also, I wanna know how to make my own enerf_outdoor dataset

To make your own ENeRF_Outdoor dataset, please capture the dynamic scene using the multiple synchronized cameras. https://github.com/zju3dv/ENeRF/blob/master/docs/enerf_outdoor.md

To generate the 3D bbox (vhull/000000.npy), you can run some background matting methods to generate the mask and then use the mask to compute visual hull for the foreground. e.g., https://github.com/PeterL1n/BackgroundMattingV2

haotongl commented 1 year ago

Hi, I try to prepare my own Mocap dataset to run your code but the youtube link in https://chingswy.github.io/easymocap-public-doc/quickstart/capture_youtube.html is private or not exist. Could you please release the multiview vedio of your Mocap vedio so that I can have a try on preparing Mocap dataset. Thanks in advance!

I am afraid that you should open an issue in EasyMocap repo to remind Dr. Shuai (https://github.com/chingswy) for the expired link. Besides, I think it would take a lot of effort to get ENeRF to work on Internet Videos.

CHDTevior commented 1 year ago

May I ask how to obtain the annots.json and the new annots.npy? I want to finetune the model using my own data set similar to zju-mocap, but I found that the annots.npy file format of the two is different, which makes it unreadable. How do I get these files to work with?