Closed marcoancona closed 4 years ago
Hi Marco,
thank you so much for your feedback! We really appreciate that. Yes, perturbation-based methods are definitely planned. Originally, we were thinking to add them in the first release but decided to postpone and focus on the gradient based approaches first. The library should be easily extensible for perturbation based approaches as well.
Thank you
@NarineK and @vivekmig are actively working on this. Closing, but we can reopen later if needed. Thank you for the question!
Hello, Kudos for the great work. I believe this has great potential. I wonder what is in your roadmap, especially regarding perturbation-based attribution methods (Occlusion, LIME/KernelSHAP, Shapley Value sampling, etc.).
Are these planned at all? While being orders of magnitude slower, these methods have the advantage that they can be applied to any black-box model (ie. any network architecture is supported out-of-the-box, with no need to instrument layers or implement custom modules). The implementation into Captum should be easier too. Moreover, Shapley Value attributions have unique theoretical properties that might be important when speed is not critical.
While it makes sense to focus on gradient-based methods first, maybe the structure of the library should be such that these methods can be easily added in the future.