search

ChenWu98 / cycle-diffusion

[ICCV 2023] A latent space for stochastic diffusion models
Other
573 stars 35 forks source link

A convolution operator error occurred when run "zero-shot image-to-image translation with text-to-image diffusion models" #32

Closed ThomasCai closed 1 month ago

ThomasCai commented 1 month ago

Thank you for your work, but I have a question about "Zero-shot image-to-image translation with text-to-image diffusion models".

My Env: ubuntu 22.04 x86 My script: (reference https://github.com/ChenWu98/cycle-diffusion?tab=readme-ov-file#usage)

torchrun --nproc_per_node 1 --master_port 1405 main.py --seed 42 --cfg experiments/translate_text2img256_stable_diffusion_stochastic_1.cfg --run_name translate_text2img256_stable_diffusion_stochastic_1_42 --logging_strategy steps --logging_first_step true --logging_steps 4 --evaluation_strategy steps --eval_steps 50 --metric_for_best_model CLIPEnergy --greater_is_better false --save_strategy steps --save_steps 50 --save_total_limit 1 --load_best_model_at_end --gradient_accumulation_steps 4 --num_train_epochs 0 --adafactor false --learning_rate 1e-3 --do_eval --output_dir output/translate_text2img256_stable_diffusion_stochastic_1_42 --overwrite_output_dir --per_device_train_batch_size 1 --per_device_eval_batch_size 4 --eval_accumulation_steps 4 --ddp_find_unused_parameters true --verbose true

stable-diffusion model use: https://github.com/ChenWu98/cycle-diffusion?tab=readme-ov-file#pre-trained-diffusion-models

No code changes

Error log:

Traceback (most recent call last):
  File "/data/thomascai/codes/cycle-diffusion/main.py", line 150, in <module>
    main()
  File "/data/thomascai/codes/cycle-diffusion/main.py", line 124, in main
    metrics = trainer.evaluate(
  File "/data/thomascai/codes/cycle-diffusion/trainer/trainer.py", line 1047, in evaluate
    metrics, num_samples = eval_loop(
  File "/data/thomascai/codes/cycle-diffusion/trainer/trainer.py", line 829, in evaluation_loop
    prediction_outputs = self.prediction_step(inputs)
  File "/data/thomascai/codes/cycle-diffusion/trainer/trainer.py", line 789, in prediction_step
    images, weighted_loss, losses = self.model(**inputs)
  File "/home/dell/miniconda3/envs/dreamda/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1518, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/home/dell/miniconda3/envs/dreamda/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1527, in _call_impl
    return forward_call(*args, **kwargs)
  File "/home/dell/miniconda3/envs/dreamda/lib/python3.9/site-packages/torch/nn/parallel/distributed.py", line 1519, in forward
    else self._run_ddp_forward(*inputs, **kwargs)
  File "/home/dell/miniconda3/envs/dreamda/lib/python3.9/site-packages/torch/nn/parallel/distributed.py", line 1355, in _run_ddp_forward
    return self.module(*inputs, **kwargs)  # type: ignore[index]
  File "/home/dell/miniconda3/envs/dreamda/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1518, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/home/dell/miniconda3/envs/dreamda/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1527, in _call_impl
    return forward_call(*args, **kwargs)
  File "/data/thomascai/codes/cycle-diffusion/model/text_unsupervised_translation.py", line 30, in forward
    z_ensemble = self.gan_wrapper.encode(image=original_image, encode_text=encode_text)
  File "/data/thomascai/codes/cycle-diffusion/model/gan_wrapper/stable_diffusion_stochastic_text_wrapper.py", line 182, in encode
    encoder_posterior = self.generator.encode_first_stage(image)
  File "/data/thomascai/codes/cycle-diffusion/model/lib/stable_diffusion/ldm/models/diffusion/ddpm.py", line 854, in encode_first_stage
    return self.first_stage_model.encode(x)
  File "/data/thomascai/codes/cycle-diffusion/model/lib/stable_diffusion/ldm/models/autoencoder.py", line 325, in encode
    h = self.encoder(x)
  File "/home/dell/miniconda3/envs/dreamda/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1518, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/home/dell/miniconda3/envs/dreamda/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1527, in _call_impl
    return forward_call(*args, **kwargs)
  File "/data/thomascai/codes/cycle-diffusion/model/lib/stable_diffusion/ldm/modules/diffusionmodules/model.py", line 442, in forward
    h = self.down[i_level].block[i_block](hs[-1], temb)
  File "/home/dell/miniconda3/envs/dreamda/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1518, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/home/dell/miniconda3/envs/dreamda/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1527, in _call_impl
    return forward_call(*args, **kwargs)
  File "/data/thomascai/codes/cycle-diffusion/model/lib/stable_diffusion/ldm/modules/diffusionmodules/model.py", line 125, in forward
    h = self.conv1(h)
  File "/home/dell/miniconda3/envs/dreamda/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1518, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/home/dell/miniconda3/envs/dreamda/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1527, in _call_impl
    return forward_call(*args, **kwargs)
  File "/home/dell/miniconda3/envs/dreamda/lib/python3.9/site-packages/torch/nn/modules/conv.py", line 460, in forward
    return self._conv_forward(input, self.weight, self.bias)
  File "/home/dell/miniconda3/envs/dreamda/lib/python3.9/site-packages/torch/nn/modules/conv.py", line 456, in _conv_forward
    return F.conv2d(input, weight, bias, self.stride,
RuntimeError: Invalid argument

Looking forward to your reply, thank you~

ThomasCai commented 1 month ago

Oh, I solved it. It was my environment. I guess it's a torch version problem.