TRAIS-Lab / dattri

`dattri` is a PyTorch library for developing, benchmarking, and deploying efficient data attribution algorithms.
https://trais-lab.github.io/dattri/
24 stars 8 forks source link

[dattri.algorithm] Adapt `vmap` to IFAttributor #81

Closed tingwl0122 closed 3 months ago

tingwl0122 commented 3 months ago

Description

This PR utilizes vmap to compute gradient within IF.

1. Motivation and Context

To speed up gradient computation in IF. (basically, borrow the idea from #72 )

2. Summary of the change

3. What tests have been added/updated for the change?

tingwl0122 commented 3 months ago

Results of running example/cifar2_resnet9/influence_function_noisy_label.py: (also similar to random checking)

Peak memory usage: 39243.571712 MB Attribution time: 136301.74650335312

Checked Data Sample Found flipped Sample
0 0
100 9
200 25
300 35
400 40
500 46
600 56
700 65
800 76
900 90

TheaperDeng commented 3 months ago

Please fix the conflict. Then I think it should be good for now.