yanqinJiang / Consistent4D

[ICLR 2024] Official Implementation of Consistent4D: Consistent 360° Dynamic Object Generation from Monocular Video
https://consistent4d.github.io/
Apache License 2.0
227 stars 6 forks source link

Export mesh problem #7

Open yufeng9819 opened 3 months ago

yufeng9819 commented 3 months ago

Hi!

Thanks for your wonderful work. Now, I am trying to export mesh from training result through the following command "python launch.py --config configs/consistent4d.yaml --export --gpu 1 data.image_seq_path=./load/demo_pac_nerf/letter system.exporter_type=mesh-exporter system.exporter.fmt=obj". However, I encounter the following error. image

I check the code in ./Consistent4D-main/threestudio/models/geometry/cascade_kplanes.py and find that it did miss the required arguments "timestamps". image So, I want to ask how to add the missing arguments to the function "forward_density".

Looking forward to your reply.

yanqinJiang commented 3 months ago

Currently only per-frame mesh could be exported, referring to #5 For your case, a simple solution is

  1. Set timesteps in forward_densify as keyword argument so that we don't need to pass the parameters.
  2. Add export_timestep in configs here and here, the value should between [-1, 1]
  3. Modify the forward_densify function:

    def forward_density(self, points: Float[Tensor, "*N Di"], timestamps: Float[Tensor, "*N 1"]=None, occ_eval=False) -> Float[Tensor, "*N 1"]:
        points_unscaled = points
        points = contract_to_unisphere(points_unscaled, self.bbox, self.unbounded)
        points = points * 2 - 1  # convert to [-1, 1] for grid sample
    
       if timestamps is None:
          timestamps = torch.ones_like(points[..., 0:1]) * self.cfg.export_timestamp
    
        points_input = torch.cat([points, timestamps], dim=-1) # timestamps range [-1, 1]
    
        # the rest lines remain the same
        # 

Please have a try, and if you meet any problem, feel free to ask me for help. (I didn't tested this code by myself)