search

pytorch / captum

Model interpretability and understanding for PyTorch
https://captum.ai
BSD 3-Clause "New" or "Revised" License
4.81k stars 489 forks source link

CCA/CKA/OPD support #572

Open brando90 opened 3 years ago

brando90 commented 3 years ago

Is CCA or CKA or Orthogonal Procrustes Distance (OPD) available for captum?

brando90 commented 3 years ago

https://discuss.pytorch.org/t/cca-cka-for-captum/107084

brando90 commented 3 years ago

https://discuss.pytorch.org/t/what-is-a-good-cca-cka-library-for-pytorch-that-works-ideally-with-gpu/104889

NarineK commented 3 years ago

@brando90, thank you for the question. No, we don't have CCA / CKA implementation in captum library. We thought about it but we haven't added it yet.

brando90 commented 3 years ago

@brando90, thank you for the question. No, we don't have CCA / CKA implementation in captum library. We thought about it but we haven't added it yet.

Thanks fo your reply Nerine!

What do you think of incorporating anatome's CCA/CKA etc into captum (since it seems to be an already working implementation of this in pytorch)?

NarineK commented 3 years ago

Thank you @brando90 for sharing the link. I'll look deeper into it and estimate the efforts required to implement it. You mentioned that currently anatome doesn't work reliably and ideally you'd like to have a more reliable implementation of the algorithms implemented there. Is that right ?

brando90 commented 3 years ago

Yes thats exactly right! There are issues like weird cuda errors though I believe its their implementation.

On Jan 20, 2021, at 4:35 PM, NarineK notifications@github.com wrote:

Thank you @brando90 https://github.com/brando90 for sharing the link. I'll look deeper into it and estimate the efforts required to implement it. You mentioned that currently anatome doesn't work reliably and ideally you'd like to have a more reliable implementation of the algorithms implemented there. Is that right ?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/pytorch/captum/issues/572#issuecomment-763997096, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAOE6LSEBO2NAIOGNP75E73S25LBVANCNFSM4VIQON5Q.

pietrolesci commented 2 years ago

Hi @NarineK,

I hope you are doing well. Any update on adding support for CCA, PWCCA, CKA, SVCCA in Captum? As mentioned by @brando90, the anatome library - his fork in particular - has some implementations already. Would be great to have it under the Captum umbrella.

Happy to help if needed.

Thanks a lot in advance for your consideration.

Best, Pietro

brando90 commented 2 years ago

Hi @NarineK,

I hope you are doing well. Any update on adding support for CCA, PWCCA, CKA, SVCCA in Captum? As mentioned by @brando90, the anatome library - his fork in particular - has some implementations already. Would be great to have it under the Captum umbrella.

Happy to help if needed.

Thanks a lot in advance for your consideration.

Best, Pietro

I've extended the original and tested it here (with all distances): https://github.com/brando90/ultimate-anatome/tree/master/sanity_checks I recommend to do that (unless you want to get in the hairy details in the original gitissue). My fork is in pypi so you can do pip install ultimate-anatome and it should work out of the box I hope. If not post issues in my repo.

brando90 commented 2 years ago

my suggestion if you want to help is probably to test it more thoroughly e.g. reproducing all tests here: https://github.com/google/svcca/blob/master/tutorials/001_Introduction.ipynb and trying it in your code. Then we could fix my fork with pull requests + add the extra sanity checks.

pietrolesci commented 2 years ago

Hi @brando90,

Thanks for your answer. I definitely think that this can be a way forward. Though, having everything in Captum was a quite captivating idea (pun intended). I will definitely use your fork for now and will monitor Captum development to see whether this will be supported.

Best, Pietro

brando90 commented 2 years ago

Though, having everything in Captum was a quite captivating idea (pun intended).

Agreed. How else do you recommend going forward? I think the best is:

  1. reproduce as many tests from google's SVCCA tutorial (to gain confidence)
  2. then once we know we have a slightly solid version of my ultimate-anatome, push it to Captum - to captive future users ;)

Let me know if you think there is a better alternative. :)

Regards, Brando

NarineK commented 2 years ago

We have done some work CCA/CKA that hasn't been merged to master yet. @bilalsal can tell more about it.

pietrolesci commented 2 years ago

Hi @NarineK,

Thanks a lot for your answer.

To add to the discussion, I have recently found out that @AntixK (CC'ed) has published the Pytorch-Model-Compare library. I think there can be value in coordinating the efforts on this topic - I hope this can be useful to centralize the discussion.

Best, Pietro

brando90 commented 2 years ago

thanks!

Not sure how we can do that, perhaps pinging them here might be best, but for now my ultimate-anatome is working and I've put enough effort into it that that's what I will stick with for now unless there are excellent reasons to switch. I agree it's a pitty for the distributed efforts, alas! I guess it happens when people are secretive about research or perhaps it's hard to google with slightly different names.

Thanks Pietro.

if it has any value, I added the links to the libraries in the end of my readme.

joshua-xia commented 1 year ago

any progress? It is very important feature, Thanks!