[Bug] Input type (c10::BFloat16) and bias type (float) should be the same

[usamaa-saleem]

epoch 1/200
/home/ubuntu/tools/sd-scripts/.venv/lib/python3.10/site-packages/torch/utils/checkpoint.py:31: UserWarning: None of the inputs have requires_grad=True. Gradients will be None
  warnings.warn("None of the inputs have requires_grad=True. Gradients will be None")
Traceback (most recent call last):
  File "/home/ubuntu/tools/sd-scripts/fine_tune.py", line 437, in <module>
    train(args)
  File "/home/ubuntu/tools/sd-scripts/fine_tune.py", line 314, in train
    noise_pred = unet(noisy_latents, timesteps, encoder_hidden_states).sample
  File "/home/ubuntu/tools/sd-scripts/.venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl
    return forward_call(*args, **kwargs)
  File "/home/ubuntu/tools/sd-scripts/.venv/lib/python3.10/site-packages/accelerate/utils/operations.py", line 490, in __call__
    return convert_to_fp32(self.model_forward(*args, **kwargs))
  File "/home/ubuntu/tools/sd-scripts/.venv/lib/python3.10/site-packages/torch/amp/autocast_mode.py", line 14, in decorate_autocast
    return func(*args, **kwargs)
  File "/home/ubuntu/tools/sd-scripts/.venv/lib/python3.10/site-packages/diffusers/models/unet_2d_condition.py", line 375, in forward
    sample = self.conv_in(sample)
  File "/home/ubuntu/tools/sd-scripts/.venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl
    return forward_call(*args, **kwargs)
  File "/home/ubuntu/tools/sd-scripts/.venv/lib/python3.10/site-packages/torch/nn/modules/conv.py", line 463, in forward
    return self._conv_forward(input, self.weight, self.bias)
  File "/home/ubuntu/tools/sd-scripts/.venv/lib/python3.10/site-packages/torch/nn/modules/conv.py", line 459, in _conv_forward
    return F.conv2d(input, weight, bias, self.stride,
RuntimeError: Input type (c10::BFloat16) and bias type (float) should be the same
steps:   0%|                                                                                                                                                                         | 0/146000 [17:24<?, ?it/s]
Traceback (most recent call last):
  File "/home/ubuntu/tools/sd-scripts/.venv/bin/accelerate", line 8, in <module>
    sys.exit(main())
  File "/home/ubuntu/tools/sd-scripts/.venv/lib/python3.10/site-packages/accelerate/commands/accelerate_cli.py", line 45, in main
    args.func(args)
  File "/home/ubuntu/tools/sd-scripts/.venv/lib/python3.10/site-packages/accelerate/commands/launch.py", line 1104, in launch_command
    simple_launcher(args)
  File "/home/ubuntu/tools/sd-scripts/.venv/lib/python3.10/site-packages/accelerate/commands/launch.py", line 567, in simple_launcher
    raise subprocess.CalledProcessError(returncode=process.returncode, cmd=cmd)
subprocess.CalledProcessError: Command '['/home/ubuntu/tools/sd-scripts/.venv/bin/python3.10', 'fine_tune.py', '--config_file', '/home/ubuntu/unsplash/training/toml_files/config.toml']' returned non-zero exit status 1.

Anyone can help with this?

I am using bf16 as my precision argument. If needed I can provide my config.toml file as well

[kohya-ss]

I hope recent update (same as #165, add with accelerator.autocast():) will fix this issue. Please check with the latest version.

[kohya-ss]

Oh might fix closes this issue...

[usamaa-saleem]

There still is the same issue @kohya-ss