Problem with unsqueeze (aka. CUDA error: out of memory)

[b4zz4]

Model loaded
Processing file: ./dataset/data/00002.jpg
Performing inversion...
Traceback (most recent call last):
  File "estimate_input_noise.py", line 70, in <module>
    outputs = invertor.perform_inversion(file_name, cond = None, init_noise_init = None, loss_weights= {'latents': 1. , 'pixels':1.} )
  File "/home/bazza/src/DIA/ddim_invertor.py", line 92, in perform_inversion
    cond = self.__tokens2conditioning(cond_out)
  File "/home/bazza/src/DIA/ddim_invertor.py", line 223, in __tokens2conditioning
    conditioning = self.ddim_sampler.model.cond_stage_model.transformer(inputs_embeds = tokens.unsqueeze(0))['last_hidden_state']
AttributeError: 'NoneType' object has no attribute 'unsqueeze'

[subrtadel]

Hi, it seems like the pickle with token inversion does not exist. Could you provide the code you ran to reproduce the problem?

[b4zz4]

Hi, Looks like pytorch overcame my 24gb video memory

I don't know why this is happening. But i don't know how to fix it.

Selected timesteps: tensor([4, 0, 5, 2, 3, 6, 7, 1])
  0%|                                                                             | 0/10 [00:06<?, ?it/s, loss=0.24, indices=tensor([4, 0, 5, 2, 3, 6, 7, 1])]
Traceback (most recent call last):
  File "estimate_CLIP_features.py", line 65, in <module>
    output = invertor.perform_cond_inversion_individual_timesteps(file_path, None, optimize_tokens=True)
  File "/home/bazza/src/DIA/ddim_invertor.py", line 290, in perform_cond_inversion_individual_timesteps
    loss.backward()
  File "/home/bazza/src/miniforge3/envs/dia_env/lib/python3.8/site-packages/torch/_tensor.py", line 363, in backward
    torch.autograd.backward(self, gradient, retain_graph, create_graph, inputs=inputs)
  File "/home/bazza/src/miniforge3/envs/dia_env/lib/python3.8/site-packages/torch/autograd/__init__.py", line 173, in backward
    Variable._execution_engine.run_backward(  # Calls into the C++ engine to run the backward pass
  File "/home/bazza/src/miniforge3/envs/dia_env/lib/python3.8/site-packages/torch/autograd/function.py", line 253, in apply
    return user_fn(self, *args)
  File "/home/bazza/src/DIA/stable-diffusion/ldm/modules/diffusionmodules/util.py", line 139, in backward
    input_grads = torch.autograd.grad(
  File "/home/bazza/src/miniforge3/envs/dia_env/lib/python3.8/site-packages/torch/autograd/__init__.py", line 275, in grad
    return Variable._execution_engine.run_backward(  # Calls into the C++ engine to run the backward pass
RuntimeError: CUDA error: out of memory
CUDA kernel errors might be asynchronously reported at some other API call,so the stacktrace below might be incorrect.
For debugging consider passing CUDA_LAUNCH_BLOCKING=1.

[b4zz4]

this generates the same result: python estimate_CLIP_features.py --input_img dataset/data/00000.png

[subrtadel]

I believe that the first issue with unsqueeze was different from the second one. I referenced the second one in a new thread. The first one was caused by trying to perform the noise inversion before the CLIP feature inversion. I added file check so now it should be clear why the error occured.