search

MarcelRobeer / ContrastiveExplanation

Contrastive Explanation (Foil Trees), developed at TNO/Utrecht University
BSD 3-Clause "New" or "Revised" License
44 stars 6 forks source link

Always getting the warning "UserWarning: Could not find a difference between fact...", with blank explenations - for any dataset, and every sample. #6

Closed mlds2020 closed 4 years ago

mlds2020 commented 4 years ago

I am trying to exactly recreate the example from the README for the Iris-dataset. Unforunately, when running .explain_instance_domain(model.predict_proba, sample) I get the following output:

[F] Picked foil "1" using foil selection strategy "second" [D] Obtaining neighborhood data C:\Users\dsemkoandrosenko\contrastive_explanation\contrastive_explanation.py:264: UserWarning: Could not find a difference between fact "setosa" and foil "versicolor" warnings.warn(f'Could not find a difference between fact ' "The model predicted 'setosa' instead of 'versicolor' because ''"

I get the same issue with every single other sample, and even every other dataset I try. What could be the issue?

Versions: Windows 10 Python: 3.7.4 Scikit-Learn: 0.21.3 Numpy: 1.18.2

MarcelRobeer commented 4 years ago

I am unable to reproduce this error. The model attempts to perturb the input you currently have into new inputs and build a local model upon this. This method indicates that within the generated neighborhood data there are not enough instances of the target class to provide an explanation. This could be due to several factors:

  1. There are not enough neighborhood data points
  2. The perturbation process is too simple for the complex data
  3. The local model does not fit well

However, seeing as it also happens on the IRIS dataset I assume none of these assumptions hold. Could you please provide me with a minimal working example that produces this error?

mlds2020 commented 4 years ago

I literally copy-pasted the code from the README.md