threestudio-project / threestudio

A unified framework for 3D content generation.
Apache License 2.0
6.32k stars 480 forks source link

TypeError: AttnProcessor2_0.__call__() got an unexpected keyword argument 'scale' #490

Closed DGdev001 closed 4 months ago

DGdev001 commented 4 months ago

Alright Iv'e been trying to get this to work with my own images for a while now, and finally got to the point where it would seemingly work.

I'm on Ubuntu 22.04 using Python 3.10.6 via pyenv, and the first two steps described on the documentation work flawlessly (NeRF & NeuS alongside Geometry Refinement)

Hardware is a Ryzen 1700 (Stock 3.0ghz) with 32GB of DDR4 (2133mhz, no XMP enabled) and a RTX 3090 with 24GB of VRAM, I'm using the "Fast Config"

This issue only arises during Stage 3 (Texture refinement) so i suspect it's a missing/conflicting dependency on my end.

i set the checkpoint after stage 2 using the command

ckpt=outputs/dreamcraft3d-geometry/$prompt@LAST/ckpts/last.ckpt

It loads Stable Diffusion and Zero123 successfully and then generates this error

/home/dgdev/.pyenv/versions/3.10.6/lib/python3.10/site-packages/pytorch_lightning/trainer/connectors/data_connector.py:424: The 'train_dataloader' does not have many workers which may be a bottleneck. Consider increasing the value of the `num_workers` argument` to `num_workers=15` in the `DataLoader` to improve performance.
/home/dgdev/.pyenv/versions/3.10.6/lib/python3.10/site-packages/pytorch_lightning/trainer/connectors/data_connector.py:424: The 'val_dataloader' does not have many workers which may be a bottleneck. Consider increasing the value of the `num_workers` argument` to `num_workers=15` in the `DataLoader` to improve performance.
Epoch 0: |                                                | 0/? [00:00<?, ?it/s]Traceback (most recent call last):
  File "/home/dgdev/threestudio/launch.py", line 301, in <module>
    main(args, extras)
  File "/home/dgdev/threestudio/launch.py", line 244, in main
    trainer.fit(system, datamodule=dm, ckpt_path=cfg.resume)
  File "/home/dgdev/.pyenv/versions/3.10.6/lib/python3.10/site-packages/pytorch_lightning/trainer/trainer.py", line 543, in fit
    call._call_and_handle_interrupt(
  File "/home/dgdev/.pyenv/versions/3.10.6/lib/python3.10/site-packages/pytorch_lightning/trainer/call.py", line 43, in _call_and_handle_interrupt
    return trainer.strategy.launcher.launch(trainer_fn, *args, trainer=trainer, **kwargs)
  File "/home/dgdev/.pyenv/versions/3.10.6/lib/python3.10/site-packages/pytorch_lightning/strategies/launchers/subprocess_script.py", line 105, in launch
    return function(*args, **kwargs)
  File "/home/dgdev/.pyenv/versions/3.10.6/lib/python3.10/site-packages/pytorch_lightning/trainer/trainer.py", line 579, in _fit_impl
    self._run(model, ckpt_path=ckpt_path)
  File "/home/dgdev/.pyenv/versions/3.10.6/lib/python3.10/site-packages/pytorch_lightning/trainer/trainer.py", line 986, in _run
    results = self._run_stage()
  File "/home/dgdev/.pyenv/versions/3.10.6/lib/python3.10/site-packages/pytorch_lightning/trainer/trainer.py", line 1030, in _run_stage
    self.fit_loop.run()
  File "/home/dgdev/.pyenv/versions/3.10.6/lib/python3.10/site-packages/pytorch_lightning/loops/fit_loop.py", line 205, in run
    self.advance()
  File "/home/dgdev/.pyenv/versions/3.10.6/lib/python3.10/site-packages/pytorch_lightning/loops/fit_loop.py", line 363, in advance
    self.epoch_loop.run(self._data_fetcher)
  File "/home/dgdev/.pyenv/versions/3.10.6/lib/python3.10/site-packages/pytorch_lightning/loops/training_epoch_loop.py", line 140, in run
    self.advance(data_fetcher)
  File "/home/dgdev/.pyenv/versions/3.10.6/lib/python3.10/site-packages/pytorch_lightning/loops/training_epoch_loop.py", line 250, in advance
    batch_output = self.automatic_optimization.run(trainer.optimizers[0], batch_idx, kwargs)
  File "/home/dgdev/.pyenv/versions/3.10.6/lib/python3.10/site-packages/pytorch_lightning/loops/optimization/automatic.py", line 190, in run
    self._optimizer_step(batch_idx, closure)
  File "/home/dgdev/.pyenv/versions/3.10.6/lib/python3.10/site-packages/pytorch_lightning/loops/optimization/automatic.py", line 268, in _optimizer_step
    call._call_lightning_module_hook(
  File "/home/dgdev/.pyenv/versions/3.10.6/lib/python3.10/site-packages/pytorch_lightning/trainer/call.py", line 159, in _call_lightning_module_hook
    output = fn(*args, **kwargs)
  File "/home/dgdev/.pyenv/versions/3.10.6/lib/python3.10/site-packages/pytorch_lightning/core/module.py", line 1308, in optimizer_step
    optimizer.step(closure=optimizer_closure)
  File "/home/dgdev/.pyenv/versions/3.10.6/lib/python3.10/site-packages/pytorch_lightning/core/optimizer.py", line 153, in step
    step_output = self._strategy.optimizer_step(self._optimizer, closure, **kwargs)
  File "/home/dgdev/.pyenv/versions/3.10.6/lib/python3.10/site-packages/pytorch_lightning/strategies/ddp.py", line 270, in optimizer_step
    optimizer_output = super().optimizer_step(optimizer, closure, model, **kwargs)
  File "/home/dgdev/.pyenv/versions/3.10.6/lib/python3.10/site-packages/pytorch_lightning/strategies/strategy.py", line 238, in optimizer_step
    return self.precision_plugin.optimizer_step(optimizer, model=model, closure=closure, **kwargs)
  File "/home/dgdev/.pyenv/versions/3.10.6/lib/python3.10/site-packages/pytorch_lightning/plugins/precision/precision.py", line 122, in optimizer_step
    return optimizer.step(closure=closure, **kwargs)
  File "/home/dgdev/.pyenv/versions/3.10.6/lib/python3.10/site-packages/torch/optim/optimizer.py", line 280, in wrapper
    out = func(*args, **kwargs)
  File "/home/dgdev/.pyenv/versions/3.10.6/lib/python3.10/site-packages/torch/optim/optimizer.py", line 33, in _use_grad
    ret = func(self, *args, **kwargs)
  File "/home/dgdev/.pyenv/versions/3.10.6/lib/python3.10/site-packages/torch/optim/adamw.py", line 148, in step
    loss = closure()
  File "/home/dgdev/.pyenv/versions/3.10.6/lib/python3.10/site-packages/pytorch_lightning/plugins/precision/precision.py", line 108, in _wrap_closure
    closure_result = closure()
  File "/home/dgdev/.pyenv/versions/3.10.6/lib/python3.10/site-packages/pytorch_lightning/loops/optimization/automatic.py", line 144, in __call__
    self._result = self.closure(*args, **kwargs)
  File "/home/dgdev/.pyenv/versions/3.10.6/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context
    return func(*args, **kwargs)
  File "/home/dgdev/.pyenv/versions/3.10.6/lib/python3.10/site-packages/pytorch_lightning/loops/optimization/automatic.py", line 129, in closure
    step_output = self._step_fn()
  File "/home/dgdev/.pyenv/versions/3.10.6/lib/python3.10/site-packages/pytorch_lightning/loops/optimization/automatic.py", line 317, in _training_step
    training_step_output = call._call_strategy_hook(trainer, "training_step", *kwargs.values())
  File "/home/dgdev/.pyenv/versions/3.10.6/lib/python3.10/site-packages/pytorch_lightning/trainer/call.py", line 311, in _call_strategy_hook
    output = fn(*args, **kwargs)
  File "/home/dgdev/.pyenv/versions/3.10.6/lib/python3.10/site-packages/pytorch_lightning/strategies/strategy.py", line 389, in training_step
    return self._forward_redirection(self.model, self.lightning_module, "training_step", *args, **kwargs)
  File "/home/dgdev/.pyenv/versions/3.10.6/lib/python3.10/site-packages/pytorch_lightning/strategies/strategy.py", line 640, in __call__
    wrapper_output = wrapper_module(*args, **kwargs)
  File "/home/dgdev/.pyenv/versions/3.10.6/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl
    return forward_call(*args, **kwargs)
  File "/home/dgdev/.pyenv/versions/3.10.6/lib/python3.10/site-packages/torch/nn/parallel/distributed.py", line 1156, in forward
    output = self._run_ddp_forward(*inputs, **kwargs)
  File "/home/dgdev/.pyenv/versions/3.10.6/lib/python3.10/site-packages/torch/nn/parallel/distributed.py", line 1110, in _run_ddp_forward
    return module_to_run(*inputs[0], **kwargs[0])  # type: ignore[index]
  File "/home/dgdev/.pyenv/versions/3.10.6/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl
    return forward_call(*args, **kwargs)
  File "/home/dgdev/.pyenv/versions/3.10.6/lib/python3.10/site-packages/pytorch_lightning/strategies/strategy.py", line 633, in wrapped_forward
    out = method(*_args, **_kwargs)
  File "/home/dgdev/threestudio/custom/threestudio-dreamcraft3D/system/dreamcraft3d.py", line 420, in training_step
    out = self.training_substep(
  File "/home/dgdev/threestudio/custom/threestudio-dreamcraft3D/system/dreamcraft3d.py", line 222, in training_substep
    guidance_out = self.guidance(
  File "/home/dgdev/.pyenv/versions/3.10.6/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl
    return forward_call(*args, **kwargs)
  File "/home/dgdev/threestudio/custom/threestudio-dreamcraft3D/guidance/stable_diffusion_bsd_lora_guidance.py", line 1175, in forward
    loss_pretrain = self.train_pretrain(
  File "/home/dgdev/threestudio/custom/threestudio-dreamcraft3D/guidance/stable_diffusion_bsd_lora_guidance.py", line 1031, in train_pretrain
    images_sample = self._sample(
  File "/home/dgdev/.pyenv/versions/3.10.6/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context
    return func(*args, **kwargs)
  File "/home/dgdev/.pyenv/versions/3.10.6/lib/python3.10/site-packages/torch/amp/autocast_mode.py", line 14, in decorate_autocast
    return func(*args, **kwargs)
  File "/home/dgdev/threestudio/custom/threestudio-dreamcraft3D/guidance/stable_diffusion_bsd_lora_guidance.py", line 418, in _sample
    noise_pred = unet(
  File "/home/dgdev/.pyenv/versions/3.10.6/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl
    return forward_call(*args, **kwargs)
  File "/home/dgdev/.pyenv/versions/3.10.6/lib/python3.10/site-packages/diffusers/models/unet_2d_condition.py", line 915, in forward
    sample, res_samples = downsample_block(
  File "/home/dgdev/.pyenv/versions/3.10.6/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl
    return forward_call(*args, **kwargs)
  File "/home/dgdev/.pyenv/versions/3.10.6/lib/python3.10/site-packages/diffusers/models/unet_2d_blocks.py", line 996, in forward
    hidden_states = attn(
  File "/home/dgdev/.pyenv/versions/3.10.6/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl
    return forward_call(*args, **kwargs)
  File "/home/dgdev/.pyenv/versions/3.10.6/lib/python3.10/site-packages/diffusers/models/transformer_2d.py", line 292, in forward
    hidden_states = block(
  File "/home/dgdev/.pyenv/versions/3.10.6/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl
    return forward_call(*args, **kwargs)
  File "/home/dgdev/.pyenv/versions/3.10.6/lib/python3.10/site-packages/diffusers/models/attention.py", line 155, in forward
    attn_output = self.attn1(
  File "/home/dgdev/.pyenv/versions/3.10.6/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl
    return forward_call(*args, **kwargs)
  File "/home/dgdev/.pyenv/versions/3.10.6/lib/python3.10/site-packages/diffusers/models/attention_processor.py", line 322, in forward
    return self.processor(
TypeError: AttnProcessor2_0.__call__() got an unexpected keyword argument 'scale'
Epoch 0: |          | 0/? [00:03<?, ?it/s]                    

I'd appreciate any assistance in resolving this error, and happy to provide additional information and/or logs!

DGdev001 commented 4 months ago

Posted in wrong repo <.<