search

cybertronai / gradient-checkpointing

Make huge neural nets fit in memory
MIT License
2.71k stars 270 forks source link

'NoneType' object has no attribute 'op' #27

Open siavash-khodadadeh opened 6 years ago

siavash-khodadadeh commented 6 years ago

I am trying to run the code for my model which uses 3d convolution and fully connected layers.

grads = gradient_memory(train_loss, self.model_variables) grads = list(zip(grads, self.model_variables))

This should give me the list as

optimizer.compute_grads(train_loss, var_list=self.model_variables)

But instead, I get:

File "gradient_checkpointing.py", line 274, in inputs_to_do_before = [d_checkpoints[r].op for r in ts] 'NoneType' object has no attribute 'op'

Can you help me with this, please?

I have set the checkpoints equal to ts_all.

Zettelkasten commented 3 years ago

I got the same error when one of the tensors in checkpoints did not have any gradient (also implicit via checkpoints='speed' or so). Removing those from checkpoints fixed the issue for me.