search

pytorch / captum

Model interpretability and understanding for PyTorch
https://captum.ai
BSD 3-Clause "New" or "Revised" License
4.93k stars 498 forks source link

Inconsistency in TCAV: Order of experimental sets #909

Closed carlomarxdk closed 1 year ago

carlomarxdk commented 2 years ago

🐛 Bug

Ordering of concepts in the experemental_sets drastically changes the output of interpret (e.g. it is not consistent)

To reproduce:

Steps to reproduce:

  1. Run the "Show cases Testing with Concept Activation Vectors (TCAV) on Imagenet Dataset and GoogleNet model" official ttutorial
  2. Change the ordering of sets
  3. Run interpret

If you change the order of experemental_sets: experimental_set_zig_dot = [[zigzagged_concept, dotted_concept, stripes_concept]] or experimental_set_zig_dot = [[dotted_concept, stripes_concept, zigzagged_concept]] you get absolutely different results. In first case, indeed zigzagged concept has the largest magnitude score; however, in the second case, the dotted concept is going to have the highest magnitude score.

Expected behaviour

I would expect that the change of the ordering would give same results.

NarineK commented 2 years ago

@carlomarxdk , thank you for pointing out to this issue. There was a bug related to the ordering of returned tcav scores. I'll send out the fix soon.

NarineK commented 2 years ago

@carlomarxdk, we just merged, #915. Let us know if this helps to address the issue