search

tensorflow / tcav

Code for the TCAV ML interpretability project
Apache License 2.0
633 stars 150 forks source link

Different TCAV scores #86

Closed jjevsikov closed 4 years ago

jjevsikov commented 4 years ago

Hi, I have a simple Keras .h5 model and I am trying to get TCAV scores for some concepts, however, I noticed that if I run TCAV multiple times (resetting the runtime before each run) on the same model, target class and the same images, I am getting different TCAV and Statistical Significance scores every time.

Is it a normal behaviour? Thank you

also-a-brain-known-as-ralph commented 4 years ago

Hey @jjevsikov.

Can you provide us with some more details on how you ran TCAV along with reproduction steps? Did you use some of our colabs as a starting point? And how did you organize your data directories for TCAV?

jjevsikov commented 4 years ago

Hey @also-a-brain-known-as-ralph

Thank you for your reply and your time!
I am using my own Keras model and all data directories are organized in the same way as TCAV Colab example. In terms of reproduction steps, I just run TCAV once, then reset the runtime. After resetting the runtime, I run TCAV again on the same model and images as before, but getting different scores.

In fact, I also tried to see if I will get different scores in _Run_TCAV_oncolab.ipynb file, and indeed I am getting different scores every time even without changing anything in the provided code itself.

To be clear with reproduction steps in _Run_TCAV_oncolab.ipynb file, they are as follows:

Each time I am getting different scores.

Please do let me know if it helps and if you encounter the same situation.

Thank you!

BeenKim commented 4 years ago

Hi @jjevsikov ,

This is a known behavior because of sklearn. We use sklearn to train the linear classifier, which gives us the CAV. sklearn uses stochastic optimization to learn linear classifiers, so you get different CAV each time. Last time I checked, sklearn is using np to do this. If this is still the case, setting random seed to be constant (https://docs.scipy.org/doc/numpy-1.15.1/reference/generated/numpy.random.seed.html) should give you consistent answer.