Understanding Black-box Predictions via Influence Functions

[YeonwooSung]

Abstract

• Use influence function to trace a model's prediction back to its training data.
• Approximation of influence function that requires gradients and Hessian vectors provides valuable information
• Useful in debugging models and detecting dataset errors

Details

• Using influence function, one can ask questions such as "What is the model parameter like when certain training data was missing/altered?" without re-training the whole model
• Useful in detecting adversarial examples
• Useful in fixing mislabeled examples by providing good candidate lists, but limited boost compared to the simple listing via highest training loss

Personal Thoughts

• Understanding neural networks is difficult because all the theoretical assumptions do not hold in non-convex, data-dependent, .. environment.
• Good approximation methods are always powerful and applicable

Link: https://arxiv.org/pdf/1703.04730.pdf Authors: Pang Wei Koh(Stanford), Percy Liang(Stanford)

[Helaly96]

Are you still interested in writing this part? Would love to discuss it with someone

[YeonwooSung]

Yeh, having discussion with others is always welcome. Please share your thoughts about this paper :)