How to export Mesh in GaussianDreamer? (Missing mandatory value: tag error)

[hayoung-jeremy]

summary

• GaussianDreamer installation was successful
• Successfully created point_cloud.ply file under outputs/gs-sds-mvdream/an_astronaut_wearing_a_blue_suit@20240329-022833/save, and last.ckpt under gs-sds-mvdream/an_astronaut_wearing_a_blue_suit@20240329-022833/ckpts
• Tried running Mesh export commands causes Missing mandatory value: tag error

reproduction of the error

• after successfully created ply and ckpt, I've run :

  python launch.py --config custom/threestudio-gaussiandreamer/configs/gaussiandreamer_mvdream.yaml --export --gpu 0 resume=outputs/gs-sds-mvdream/an_astronaut_wearing_a_blue_suit@20240329-022833/ckpts/last.ckpt system.exporter_type=mesh-exporter system.geometry.isosurface_method=mc-cpu system.geometry.isosurface_resolution=256

• the error message shows :

  Import times for custom modules:
     0.0 seconds (IMPORT FAILED): custom/shap-e
     0.0 seconds: custom/threestudio-mvdream
     0.1 seconds: custom/threestudio-dreamcraft3D
     0.1 seconds: custom/threestudio-shap-e
     0.6 seconds: custom/threestudio-lrm
     1.0 seconds: custom/threestudio-gaussiandreamer
  
  Traceback (most recent call last):
    File "/home/dreamer/threestudio/launch.py", line 301, in <module>
      main(args, extras)
    File "/home/dreamer/threestudio/launch.py", line 163, in main
      cfg = load_config(args.config, cli_args=extras, n_gpus=n_gpus)
    File "/home/dreamer/threestudio/threestudio/utils/config.py", line 113, in load_config
      scfg = parse_structured(ExperimentConfig, cfg)
    File "/home/dreamer/threestudio/threestudio/utils/config.py", line 127, in parse_structured
      scfg = OmegaConf.structured(fields(**cfg))
    File "/home/dreamer/.local/lib/python3.10/site-packages/omegaconf/dictconfig.py", line 375, in __getitem__
      self._format_and_raise(key=key, value=None, cause=e)
    File "/home/dreamer/.local/lib/python3.10/site-packages/omegaconf/base.py", line 231, in _format_and_raise
      format_and_raise(
    File "/home/dreamer/.local/lib/python3.10/site-packages/omegaconf/_utils.py", line 899, in format_and_raise
      _raise(ex, cause)
    File "/home/dreamer/.local/lib/python3.10/site-packages/omegaconf/_utils.py", line 797, in _raise
      raise ex.with_traceback(sys.exc_info()[2])  # set env var OC_CAUSE=1 for full trace
    File "/home/dreamer/.local/lib/python3.10/site-packages/omegaconf/dictconfig.py", line 369, in __getitem__
      return self._get_impl(key=key, default_value=_DEFAULT_MARKER_)
    File "/home/dreamer/.local/lib/python3.10/site-packages/omegaconf/dictconfig.py", line 451, in _get_impl
      return self._resolve_with_default(
    File "/home/dreamer/.local/lib/python3.10/site-packages/omegaconf/basecontainer.py", line 96, in _resolve_with_default
      raise MissingMandatoryValue("Missing mandatory value: $FULL_KEY")
  omegaconf.errors.MissingMandatoryValue: Missing mandatory value: tag
      full_key: tag
      object_type=dict

[bennyguo]

Please try setting --config as outputs/gs-sds-mvdream/an_astronaut_wearing_a_blue_suit@20240329-022833/configs/parsed.yaml.

[hayoung-jeremy]

Thanks, but encountered another error :

• command :

  python launch.py --config outputs/gs-sds-mvdream/an_astronaut_wearing_a_blue_suit@20240329-022833/configs/parsed.yaml --export --gpu 0 resume=outputs/gs-sds-mvdream/an_astronaut_wearing_a_blue_suit@20240329-022833/ckpts/last.ckpt system.exporter_type=mesh-exporter system.geometry.isosurface_method=mc-cpu system.geometry.isosurface_resolution=256

• error :

  Traceback (most recent call last):
    File "/home/dreamer/threestudio/launch.py", line 301, in <module>
      main(args, extras)
    File "/home/dreamer/threestudio/launch.py", line 169, in main
      system: BaseSystem = threestudio.find(cfg.system_type)(
    File "/home/dreamer/threestudio/threestudio/systems/base.py", line 51, in __init__
      self.configure()
    File "/home/dreamer/threestudio/custom/threestudio-gaussiandreamer/system/gaussian_mvdream.py", line 25, in configure
      super().configure()
    File "/home/dreamer/threestudio/threestudio/systems/base.py", line 292, in configure
      self.geometry = threestudio.find(self.cfg.geometry_type)(self.cfg.geometry)
    File "/home/dreamer/threestudio/threestudio/utils/base.py", line 100, in __init__
      self.cfg = parse_structured(self.Config, cfg)
    File "/home/dreamer/threestudio/threestudio/utils/config.py", line 127, in parse_structured
      scfg = OmegaConf.structured(fields(**cfg))
  TypeError: GaussianBaseModel.Config.__init__() got an unexpected keyword argument 'isosurface_method'

• So I removed system.geometry.isosurface_method and system.geometry.isosurface_resolution, and added system.exporter.context_type=cuda run :

  python launch.py --config outputs/gs-sds-mvdream/an_astronaut_wearing_a_blue_suit@20240329-022833/configs/parsed.yaml --export --gpu 0 resume=outputs/gs-sds-mvdream/an_astronaut_wearing_a_blue_suit@20240329-022833/ckpts/last.ckpt system.exporter_type=mesh-exporter system.exporter.context_type=cuda

• and got the following error :

  Traceback (most recent call last):
    File "/home/dreamer/threestudio/launch.py", line 301, in <module>
      main(args, extras)
    File "/home/dreamer/threestudio/launch.py", line 259, in main
      trainer.predict(system, datamodule=dm, ckpt_path=cfg.resume)
    File "/home/dreamer/.local/lib/python3.10/site-packages/pytorch_lightning/trainer/trainer.py", line 864, in predict
      return call._call_and_handle_interrupt(
    File "/home/dreamer/.local/lib/python3.10/site-packages/pytorch_lightning/trainer/call.py", line 44, in _call_and_handle_interrupt
      return trainer_fn(*args, **kwargs)
    File "/home/dreamer/.local/lib/python3.10/site-packages/pytorch_lightning/trainer/trainer.py", line 903, in _predict_impl
      results = self._run(model, ckpt_path=ckpt_path)
    File "/home/dreamer/.local/lib/python3.10/site-packages/pytorch_lightning/trainer/trainer.py", line 987, in _run
      results = self._run_stage()
    File "/home/dreamer/.local/lib/python3.10/site-packages/pytorch_lightning/trainer/trainer.py", line 1028, in _run_stage
      return self.predict_loop.run()
    File "/home/dreamer/.local/lib/python3.10/site-packages/pytorch_lightning/loops/utilities.py", line 182, in _decorator
      return loop_run(self, *args, **kwargs)
    File "/home/dreamer/.local/lib/python3.10/site-packages/pytorch_lightning/loops/prediction_loop.py", line 130, in run
      return self.on_run_end()
    File "/home/dreamer/.local/lib/python3.10/site-packages/pytorch_lightning/loops/prediction_loop.py", line 202, in on_run_end
      results = self._on_predict_epoch_end()
    File "/home/dreamer/.local/lib/python3.10/site-packages/pytorch_lightning/loops/prediction_loop.py", line 368, in _on_predict_epoch_end    call._call_lightning_module_hook(trainer, "on_predict_epoch_end")
    File "/home/dreamer/.local/lib/python3.10/site-packages/pytorch_lightning/trainer/call.py", line 157, in _call_lightning_module_hook
      output = fn(*args, **kwargs)
    File "/home/dreamer/threestudio/threestudio/systems/base.py", line 332, in on_predict_epoch_end
      exporter_output: List[ExporterOutput] = self.exporter()
    File "/home/dreamer/threestudio/threestudio/models/exporters/mesh_exporter.py", line 44, in __call__
      mesh: Mesh = self.geometry.isosurface()
    File "/home/dreamer/.local/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1688, in __getattr__
      raise AttributeError(f"'{type(self).__name__}' object has no attribute '{name}'")
  AttributeError: 'GaussianBaseModel' object has no attribute 'isosurface'

[bennyguo]

If so, it means that this Gaussian model does not support mesh extraction :(

[codyreading]

Hey @hayoung-jeremy

You need to remove system.exporter_type=mesh-exporter from your command.

If you take a look at the gaussiandreamer_mvdream.yaml config it uses gaussiandreamer-mesh-exporter

Although now I'm getting this error now:

Traceback (most recent call last):
  File "/home/cra80/Projects/threestudio/launch.py", line 332, in <module>
    main(args, extras)
  File "/home/cra80/Projects/threestudio/launch.py", line 290, in main
    trainer.predict(system, datamodule=dm, ckpt_path=cfg.resume)
  File "/home/cra80/miniconda3/envs/threestudio-torch222/lib/python3.10/site-packages/pytorch_lightning/trainer/trainer.py", line 864, in predict
    return call._call_and_handle_interrupt(
  File "/home/cra80/miniconda3/envs/threestudio-torch222/lib/python3.10/site-packages/pytorch_lightning/trainer/call.py", line 44, in _call_and_handle_interrupt
    return trainer_fn(*args, **kwargs)
  File "/home/cra80/miniconda3/envs/threestudio-torch222/lib/python3.10/site-packages/pytorch_lightning/trainer/trainer.py", line 903, in _predict_impl
    results = self._run(model, ckpt_path=ckpt_path)
  File "/home/cra80/miniconda3/envs/threestudio-torch222/lib/python3.10/site-packages/pytorch_lightning/trainer/trainer.py", line 987, in _run
    results = self._run_stage()
  File "/home/cra80/miniconda3/envs/threestudio-torch222/lib/python3.10/site-packages/pytorch_lightning/trainer/trainer.py", line 1028, in _run_stage
    return self.predict_loop.run()
  File "/home/cra80/miniconda3/envs/threestudio-torch222/lib/python3.10/site-packages/pytorch_lightning/loops/utilities.py", line 182, in _decorator
    return loop_run(self, *args, **kwargs)
  File "/home/cra80/miniconda3/envs/threestudio-torch222/lib/python3.10/site-packages/pytorch_lightning/loops/prediction_loop.py", line 130, in run
    return self.on_run_end()
  File "/home/cra80/miniconda3/envs/threestudio-torch222/lib/python3.10/site-packages/pytorch_lightning/loops/prediction_loop.py", line 202, in on_run_end
    results = self._on_predict_epoch_end()
  File "/home/cra80/miniconda3/envs/threestudio-torch222/lib/python3.10/site-packages/pytorch_lightning/loops/prediction_loop.py", line 368, in _on_predict_epoch_end
    call._call_lightning_module_hook(trainer, "on_predict_epoch_end")
  File "/home/cra80/miniconda3/envs/threestudio-torch222/lib/python3.10/site-packages/pytorch_lightning/trainer/call.py", line 157, in _call_lightning_module_hook
    output = fn(*args, **kwargs)
  File "/home/cra80/Projects/threestudio/threestudio/systems/base.py", line 335, in on_predict_epoch_end
    exporter_output: List[ExporterOutput] = self.exporter()
  File "/home/cra80/Projects/threestudio/custom/threestudio-gaussiandreamer/geometry/exporter.py", line 35, in __call__
    mesh: Mesh = self.geometry.extract_mesh()
  File "/home/cra80/Projects/threestudio/custom/threestudio-gaussiandreamer/geometry/gaussian_io.py", line 268, in extract_mesh
    occ = self.extract_fields(resolution).detach().cpu().numpy()
  File "/home/cra80/miniconda3/envs/threestudio-torch222/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context
    return func(*args, **kwargs)
  File "/home/cra80/Projects/threestudio/custom/threestudio-gaussiandreamer/geometry/gaussian_io.py", line 247, in extract_fields
    w = gaussian_3d_coeff(
NameError: name 'gaussian_3d_coeff' is not defined

[codyreading]

Fixed in here: https://github.com/cxh0519/threestudio-gaussiandreamer/pull/8