`CLIPModelOutput` gradient computation wrong dimensionality

MadryLab / trak

A fast, effective data attribution method for neural networks in PyTorch

https://trak.csail.mit.edu/

MIT License

180 stars 25 forks source link

`CLIPModelOutput` gradient computation wrong dimensionality #73

Closed dwahdany closed 2 months ago

dwahdany commented 2 months ago

The function CLIPModelOutput.get_out_to_loss_grad outputs a [batch_size]-shaped tensor, but it should be (for consistency with other methods and compatibility with saver.current_store["out_to_loss"]) [batch_size, 1]-shaped. All other models have a unsqueeze(-1) at the end, for CLIP it seems to be missing.

kristian-georgiev commented 2 months ago

resolved by https://github.com/MadryLab/trak/pull/72; thanks @dwahdany!