HeliosZhao / Animate124

Animate124: Animating One Image to 4D Dynamic Scene
Apache License 2.0
151 stars 5 forks source link

Is it possible to generate a 3d mesh model, for example a .dae file? #3

Open andytriboletti opened 5 months ago

andytriboletti commented 5 months ago

I was able to run all 3 steps on my PC after reducing some of the max steps. I used threestudio. I want to include animated models in a game, not just have the video, have a .dae file that you can import and use. Is that possible?

andytriboletti commented 5 months ago

I ran this command after all the other steps:

seed=0 gpu=0 exp_root_dir=outputs DATA_DIR="panda-dance" STATIC_PROMPT="a high resolution DSLR image of panda" DYNAMIC_PROMPT="a panda is dancing" CN_PROMPT="a is dancing"

--------- Stage 2 (Semantic Refinement Stage) ---------

ckpt=outputs/animate124-stage2/${DYNAMIC_PROMPT}@LAST/ckpts/last.ckpt python launch.py --config custom/threestudio-animate124/configs/animate124-stage3-ms.yaml --export --gpu $gpu system.exporter_type=mesh-exporter system.exporter.context_type=cuda \ data.image.image_path=custom/threestudio-animate124/load/${DATA_DIR}/_rgba.png \ system.prompt_processor.prompt="${DYNAMIC_PROMPT}" \ system.prompt_processor_cn.prompt="${CN_PROMPT}" \ system.prompt_processor_cn.learned_embeds_path=custom/threestudio-animate124/load/${DATA_DIR}/learned_embeds.bin \ system.weights="$ckpt"

I had to add to base.py: out.params.pop('map_Pm', None) out.params.pop('map_Pr', None)

I was able to view a panda with the textures loaded as an .obj file. However it wasn't animating.

HeliosZhao commented 5 months ago

Good question, currently we do not support exporting a mesh with animation. Only static mesh (.obj) can be exported.

andytriboletti commented 4 months ago

Thank you for the answer. Any plans to support exporting a mesh with animation?