jina-ai / discoart

🪩 Create Disco Diffusion artworks in one line
Other
3.84k stars 249 forks source link

discoart - ERROR - no valid convolution algorithms available in CuDNN #77

Closed MaxOrtGit closed 2 years ago

MaxOrtGit commented 2 years ago

I get this error when running at default settings but at a lower res it works. I have a 3060 laptop GPU. Is there any way to either fix this or changes in the config/model to get it to run?

erorr code:

2022-07-24 08:34:25,235 - discoart - ERROR - no valid convolution algorithms available in CuDNN
Traceback (most recent call last):
  File "C:\Users\maxro\AppData\Local\Programs\Python\Python310\lib\site-packages\discoart\create.py", line 205, in create
    do_run(
  File "C:\Users\maxro\AppData\Local\Programs\Python\Python310\lib\site-packages\discoart\runner.py", line 357, in do_run
    for j, sample in enumerate(samples):
  File "C:\Users\maxro\AppData\Local\Programs\Python\Python310\lib\site-packages\guided_diffusion\gaussian_diffusion.py", line 897, in ddim_sample_loop_progressive
    out = sample_fn(
  File "C:\Users\maxro\AppData\Local\Programs\Python\Python310\lib\site-packages\guided_diffusion\gaussian_diffusion.py", line 674, in ddim_sample
    out = self.condition_score(cond_fn, out_orig, x, t, model_kwargs=model_kwargs)
  File "C:\Users\maxro\AppData\Local\Programs\Python\Python310\lib\site-packages\guided_diffusion\respace.py", line 102, in condition_score
    return super().condition_score(self._wrap_model(cond_fn), *args, **kwargs)
  File "C:\Users\maxro\AppData\Local\Programs\Python\Python310\lib\site-packages\guided_diffusion\gaussian_diffusion.py", line 399, in condition_score
    eps = eps - (1 - alpha_bar).sqrt() * cond_fn(
  File "C:\Users\maxro\AppData\Local\Programs\Python\Python310\lib\site-packages\guided_diffusion\respace.py", line 128, in __call__
    return self.model(x, new_ts, **kwargs)
  File "C:\Users\maxro\AppData\Local\Programs\Python\Python310\lib\site-packages\discoart\runner.py", line 250, in cond_fn
    torch.autograd.grad(
  File "C:\Users\maxro\AppData\Local\Programs\Python\Python310\lib\site-packages\torch\autograd\__init__.py", line 276, in grad
    return Variable._execution_engine.run_backward(  # Calls into the C++ engine to run the backward pass
RuntimeError: no valid convolution algorithms available in CuDNN
hanxiao commented 2 years ago

this error means OOM, not a bug of DiscoArt. The error msg may be misleading but it is also how pytorch throws 🤷 so upstream problem on fixing this misleading info.

since you dont tell me your VRAM, i can only guess it is very small

so try:

MaxOrtGit commented 2 years ago

my VRAM is only 6GB But I figured it out, decreasing cutn_batches to 1 got it to work, I will update with other settings that might help other people.

MaxOrtGit commented 2 years ago

well it works for a bit but after a certain amount of steps (ranging from 30-300 it will stop saying it couldn't find path (name.svg) and it was canceled even if I never touched the keyboard.

hanxiao commented 2 years ago

couldn't find path (name.svg)

version too old, plz update

but after a certain amount of steps (ranging from 30-300

because your cuts increases on later steps, check your cut schedules

MaxOrtGit commented 2 years ago

I am on 0.7.14 and still having the issue. It says Your n_backup=1 so supposedly 1 images will be generated, but only 0 images were fully completed. This may be due to the following reasons: (don't apply). To avoid this, you can set n_batches to a smaller number in create(), say create(n_batches=1) I assume I am just running out of VRAM. If so what clip_model(s) so you recommend me using for the best tradeoff of quality to performance? Also how to I check/change my cut schedules.

MaxOrtGit commented 2 years ago

Also if I try to reuse it as initial state it fails on the first one and says the same thing