search

deepseek-ai / DreamCraft3D

[ICLR 2024] Official implementation of DreamCraft3D: Hierarchical 3D Generation with Bootstrapped Diffusion Prior
https://mrtornado24.github.io/DreamCraft3D/
MIT License
1.88k stars 81 forks source link

TypeError: AttnProcessor2_0.__call__() got an unexpected keyword argument 'scale' #27

Open liuwei opened 6 months ago

liuwei commented 6 months ago

Hey, i try to run Stage 3 (Texture Refinement), After running

ckpt=outputs/dreamcraft3d-geometry/$prompt@LAST/ckpts/last.ckpt
python launch.py --config configs/dreamcraft3d-texture.yaml --train system.prompt_processor.prompt="$prompt" data.image_path="$image_path" system.geometry_convert_from="$ckpt"

i get this output resulting with this error : TypeError: AttnProcessor2_0.call() got an unexpected keyword argument 'scale'

Here is the whole log of the process

Seed set to 0
[INFO] Initializing geometry from a given checkpoint ...
[INFO] Loading Stable Diffusion ...
Loading pipeline components...: 100%|█████████████████████████████████████████████████████████| 4/4 [00:01<00:00,  2.15it/s]
Loading pipeline components...: 100%|█████████████████████████████████████████████████████████| 4/4 [00:01<00:00,  2.44it/s]
[INFO] Loaded Stable Diffusion!
[INFO] Using prompt [A girl holding a basketball wearing sportswear and a sports cap] and negative prompt []
[INFO] Using view-dependent prompts [side]:[A girl holding a basketball wearing sportswear and a sports cap, side view] [front]:[A girl holding a basketball wearing sportswear and a sports cap, front view] [back]:[A girl holding a basketball wearing sportswear and a sports cap, back view] [overhead]:[A girl holding a basketball wearing sportswear and a sports cap, overhead view]
/root/miniconda3/envs/dreamcraft3d/lib/python3.10/site-packages/torchvision/models/_utils.py:208: UserWarning: The parameter 'pretrained' is deprecated since 0.13 and may be removed in the future, please use 'weights' instead.
  warnings.warn(
/root/miniconda3/envs/dreamcraft3d/lib/python3.10/site-packages/torchvision/models/_utils.py:223: UserWarning: Arguments other than a weight enum or `None` for 'weights' are deprecated since 0.13 and may be removed in the future. The current behavior is equivalent to passing `weights=VGG16_Weights.IMAGENET1K_V1`. You can also use `weights=VGG16_Weights.DEFAULT` to get the most up-to-date weights.
  warnings.warn(msg)
loaded pretrained LPIPS loss from threestudio/utils/lpips/vgg.pth
[INFO] GPU available: True (cuda), used: True
[INFO] TPU available: False, using: 0 TPU cores
[INFO] IPU available: False, using: 0 IPUs
[INFO] HPU available: False, using: 0 HPUs
[rank: 0] Seed set to 0
Initializing distributed: GLOBAL_RANK: 0, MEMBER: 1/1
[INFO] ----------------------------------------------------------------------------------------------------
distributed_backend=nccl
All distributed processes registered. Starting with 1 processes
----------------------------------------------------------------------------------------------------

[INFO] You are using a CUDA device ('NVIDIA A100-PCIE-40GB') that has Tensor Cores. To properly utilize them, you should set `torch.set_float32_matmul_precision('medium' | 'high')` which will trade-off precision for performance. For more details, read https://pytorch.org/docs/stable/generated/torch.set_float32_matmul_precision.html#torch.set_float32_matmul_precision
[INFO] single image dataset: load image load/images/girl_rgba_rgba.png torch.Size([1, 1024, 1024, 3])
[INFO] single image dataset: load image load/images/girl_rgba_rgba.png torch.Size([1, 1024, 1024, 3])
[INFO] LOCAL_RANK: 0 - CUDA_VISIBLE_DEVICES: [0]
[INFO] 
  | Name       | Type                       | Params
----------------------------------------------------------
0 | geometry   | TetrahedraSDFGrid          | 12.6 M
1 | material   | NoMaterial                 | 0     
2 | background | SolidColorBackground       | 0     
3 | renderer   | NVDiffRasterizer           | 0     
4 | guidance   | StableDiffusionBSDGuidance | 1.7 B 
----------------------------------------------------------
1.7 B     Trainable params
0         Non-trainable params
1.7 B     Total params
6,984.340 Total estimated model params size (MB)
[INFO] Validation results will be saved to outputs/dreamcraft3d-texture/A_girl_holding_a_basketball_wearing_sportswear_and_a_sports_cap@20231227-003242/save
/root/miniconda3/envs/dreamcraft3d/lib/python3.10/site-packages/pytorch_lightning/trainer/connectors/data_connector.py:441: The 'train_dataloader' does not have many workers which may be a bottleneck. Consider increasing the value of the `num_workers` argument` to `num_workers=79` in the `DataLoader` to improve performance.
/root/miniconda3/envs/dreamcraft3d/lib/python3.10/site-packages/pytorch_lightning/trainer/connectors/data_connector.py:441: The 'val_dataloader' does not have many workers which may be a bottleneck. Consider increasing the value of the `num_workers` argument` to `num_workers=79` in the `DataLoader` to improve performance.
Epoch 0: |                                                                                            | 0/? [00:00<?, ?it/s]Traceback (most recent call last):
  File "/root/data/lw/DreamCraft3D/launch.py", line 252, in <module>
    main(args, extras)
  File "/root/data/lw/DreamCraft3D/launch.py", line 195, in main
    trainer.fit(system, datamodule=dm, ckpt_path=cfg.resume)
  File "/root/miniconda3/envs/dreamcraft3d/lib/python3.10/site-packages/pytorch_lightning/trainer/trainer.py", line 544, in fit
    call._call_and_handle_interrupt(
  File "/root/miniconda3/envs/dreamcraft3d/lib/python3.10/site-packages/pytorch_lightning/trainer/call.py", line 43, in _call_and_handle_interrupt
    return trainer.strategy.launcher.launch(trainer_fn, *args, trainer=trainer, **kwargs)
  File "/root/miniconda3/envs/dreamcraft3d/lib/python3.10/site-packages/pytorch_lightning/strategies/launchers/subprocess_script.py", line 102, in launch
    return function(*args, **kwargs)
  File "/root/miniconda3/envs/dreamcraft3d/lib/python3.10/site-packages/pytorch_lightning/trainer/trainer.py", line 580, in _fit_impl
    self._run(model, ckpt_path=ckpt_path)
  File "/root/miniconda3/envs/dreamcraft3d/lib/python3.10/site-packages/pytorch_lightning/trainer/trainer.py", line 989, in _run
    results = self._run_stage()
  File "/root/miniconda3/envs/dreamcraft3d/lib/python3.10/site-packages/pytorch_lightning/trainer/trainer.py", line 1035, in _run_stage
    self.fit_loop.run()
  File "/root/miniconda3/envs/dreamcraft3d/lib/python3.10/site-packages/pytorch_lightning/loops/fit_loop.py", line 202, in run
    self.advance()
  File "/root/miniconda3/envs/dreamcraft3d/lib/python3.10/site-packages/pytorch_lightning/loops/fit_loop.py", line 359, in advance
    self.epoch_loop.run(self._data_fetcher)
  File "/root/miniconda3/envs/dreamcraft3d/lib/python3.10/site-packages/pytorch_lightning/loops/training_epoch_loop.py", line 136, in run
    self.advance(data_fetcher)
  File "/root/miniconda3/envs/dreamcraft3d/lib/python3.10/site-packages/pytorch_lightning/loops/training_epoch_loop.py", line 240, in advance
    batch_output = self.automatic_optimization.run(trainer.optimizers[0], batch_idx, kwargs)
  File "/root/miniconda3/envs/dreamcraft3d/lib/python3.10/site-packages/pytorch_lightning/loops/optimization/automatic.py", line 187, in run
    self._optimizer_step(batch_idx, closure)
  File "/root/miniconda3/envs/dreamcraft3d/lib/python3.10/site-packages/pytorch_lightning/loops/optimization/automatic.py", line 265, in _optimizer_step
    call._call_lightning_module_hook(
  File "/root/miniconda3/envs/dreamcraft3d/lib/python3.10/site-packages/pytorch_lightning/trainer/call.py", line 157, in _call_lightning_module_hook
    output = fn(*args, **kwargs)
  File "/root/miniconda3/envs/dreamcraft3d/lib/python3.10/site-packages/pytorch_lightning/core/module.py", line 1291, in optimizer_step
    optimizer.step(closure=optimizer_closure)
  File "/root/miniconda3/envs/dreamcraft3d/lib/python3.10/site-packages/pytorch_lightning/core/optimizer.py", line 151, in step
    step_output = self._strategy.optimizer_step(self._optimizer, closure, **kwargs)
  File "/root/miniconda3/envs/dreamcraft3d/lib/python3.10/site-packages/pytorch_lightning/strategies/ddp.py", line 265, in optimizer_step
    optimizer_output = super().optimizer_step(optimizer, closure, model, **kwargs)
  File "/root/miniconda3/envs/dreamcraft3d/lib/python3.10/site-packages/pytorch_lightning/strategies/strategy.py", line 230, in optimizer_step
    return self.precision_plugin.optimizer_step(optimizer, model=model, closure=closure, **kwargs)
  File "/root/miniconda3/envs/dreamcraft3d/lib/python3.10/site-packages/pytorch_lightning/plugins/precision/precision.py", line 117, in optimizer_step
    return optimizer.step(closure=closure, **kwargs)
  File "/root/miniconda3/envs/dreamcraft3d/lib/python3.10/site-packages/torch/optim/optimizer.py", line 280, in wrapper
    out = func(*args, **kwargs)
  File "/root/miniconda3/envs/dreamcraft3d/lib/python3.10/site-packages/torch/optim/optimizer.py", line 33, in _use_grad
    ret = func(self, *args, **kwargs)
  File "/root/miniconda3/envs/dreamcraft3d/lib/python3.10/site-packages/torch/optim/adamw.py", line 148, in step
    loss = closure()
  File "/root/miniconda3/envs/dreamcraft3d/lib/python3.10/site-packages/pytorch_lightning/plugins/precision/precision.py", line 104, in _wrap_closure
    closure_result = closure()
  File "/root/miniconda3/envs/dreamcraft3d/lib/python3.10/site-packages/pytorch_lightning/loops/optimization/automatic.py", line 140, in __call__
    self._result = self.closure(*args, **kwargs)
  File "/root/miniconda3/envs/dreamcraft3d/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context
    return func(*args, **kwargs)
  File "/root/miniconda3/envs/dreamcraft3d/lib/python3.10/site-packages/pytorch_lightning/loops/optimization/automatic.py", line 126, in closure
    step_output = self._step_fn()
  File "/root/miniconda3/envs/dreamcraft3d/lib/python3.10/site-packages/pytorch_lightning/loops/optimization/automatic.py", line 315, in _training_step
    training_step_output = call._call_strategy_hook(trainer, "training_step", *kwargs.values())
  File "/root/miniconda3/envs/dreamcraft3d/lib/python3.10/site-packages/pytorch_lightning/trainer/call.py", line 309, in _call_strategy_hook
    output = fn(*args, **kwargs)
  File "/root/miniconda3/envs/dreamcraft3d/lib/python3.10/site-packages/pytorch_lightning/strategies/strategy.py", line 381, in training_step
    return self._forward_redirection(self.model, self.lightning_module, "training_step", *args, **kwargs)
  File "/root/miniconda3/envs/dreamcraft3d/lib/python3.10/site-packages/pytorch_lightning/strategies/strategy.py", line 633, in __call__
    wrapper_output = wrapper_module(*args, **kwargs)
  File "/root/miniconda3/envs/dreamcraft3d/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl
    return forward_call(*args, **kwargs)
  File "/root/miniconda3/envs/dreamcraft3d/lib/python3.10/site-packages/torch/nn/parallel/distributed.py", line 1156, in forward
    output = self._run_ddp_forward(*inputs, **kwargs)
  File "/root/miniconda3/envs/dreamcraft3d/lib/python3.10/site-packages/torch/nn/parallel/distributed.py", line 1110, in _run_ddp_forward
    return module_to_run(*inputs[0], **kwargs[0])  # type: ignore[index]
  File "/root/miniconda3/envs/dreamcraft3d/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl
    return forward_call(*args, **kwargs)
  File "/root/miniconda3/envs/dreamcraft3d/lib/python3.10/site-packages/pytorch_lightning/strategies/strategy.py", line 626, in wrapped_forward
    out = method(*_args, **_kwargs)
  File "/root/data/lw/DreamCraft3D/threestudio/systems/dreamcraft3d.py", line 367, in training_step
    out = self.training_substep(batch, batch_idx, guidance="guidance", render_type=render_type)
  File "/root/data/lw/DreamCraft3D/threestudio/systems/dreamcraft3d.py", line 196, in training_substep
    guidance_out = self.guidance(
  File "/root/miniconda3/envs/dreamcraft3d/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl
    return forward_call(*args, **kwargs)
  File "/root/data/lw/DreamCraft3D/threestudio/models/guidance/stable_diffusion_bsd_guidance.py", line 1086, in forward
    loss_pretrain = self.train_pretrain(latents, text_embeddings_vd, camera_condition, sample_new_img=sample_new_img)
  File "/root/data/lw/DreamCraft3D/threestudio/models/guidance/stable_diffusion_bsd_guidance.py", line 951, in train_pretrain
    images_sample = self._sample(
  File "/root/miniconda3/envs/dreamcraft3d/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context
    return func(*args, **kwargs)
  File "/root/miniconda3/envs/dreamcraft3d/lib/python3.10/site-packages/torch/amp/autocast_mode.py", line 14, in decorate_autocast
    return func(*args, **kwargs)
  File "/root/data/lw/DreamCraft3D/threestudio/models/guidance/stable_diffusion_bsd_guidance.py", line 394, in _sample
    noise_pred = unet(
  File "/root/miniconda3/envs/dreamcraft3d/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl
    return forward_call(*args, **kwargs)
  File "/root/miniconda3/envs/dreamcraft3d/lib/python3.10/site-packages/diffusers/models/unet_2d_condition.py", line 915, in forward
    sample, res_samples = downsample_block(
  File "/root/miniconda3/envs/dreamcraft3d/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl
    return forward_call(*args, **kwargs)
  File "/root/miniconda3/envs/dreamcraft3d/lib/python3.10/site-packages/diffusers/models/unet_2d_blocks.py", line 996, in forward
    hidden_states = attn(
  File "/root/miniconda3/envs/dreamcraft3d/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl
    return forward_call(*args, **kwargs)
  File "/root/miniconda3/envs/dreamcraft3d/lib/python3.10/site-packages/diffusers/models/transformer_2d.py", line 292, in forward
    hidden_states = block(
  File "/root/miniconda3/envs/dreamcraft3d/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl
    return forward_call(*args, **kwargs)
  File "/root/miniconda3/envs/dreamcraft3d/lib/python3.10/site-packages/diffusers/models/attention.py", line 155, in forward
    attn_output = self.attn1(
  File "/root/miniconda3/envs/dreamcraft3d/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl
    return forward_call(*args, **kwargs)
  File "/root/miniconda3/envs/dreamcraft3d/lib/python3.10/site-packages/diffusers/models/attention_processor.py", line 322, in forward
    return self.processor(
TypeError: AttnProcessor2_0.__call__() got an unexpected keyword argument 'scale'
Epoch 0: |          | 0/? [00:03<?, ?it/s]
xiaobiaodu commented 6 months ago

I also meet this problem. Hope this problem will be fixed

csdaiwei commented 6 months ago

I also meet this problem, and fixed by pip install diffusers==0.23.0

boltron1 commented 4 months ago

I also meet this problem, and fixed by pip install diffusers==0.23.0

thanks, this also worked for me.